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ABSTRACT 



The Connecticut Academy for Education implemented the School 
Improvement Support Team (SIST) in the Waterbury, Connecticut , Public Schools 
curing the 1998-99 school year. The main goal of the SijT is to provide a 
comprehensive and cohesive set of programs and activities for improving 
student performance in the Waterbury Public Schools. The Connecticut Academy 
for Education had provided assistance with curriculum development to the 
school system for the 3 previous years. The SIST's educational consultants 
worked with administrators and teachers in seven elementary schools . With a 
specific focus on assessment, mathematics, and language arts, the SIST 
provided strategic data-based technical assistance to teachers to foster 
academic achievement. The preliminary evaluation indicates that the SIST 
offers the Waterbury schools a new and innovative approach to improving 
program areas, as evidenced by the recent inclusion of 7.5 additional days in 
the teachers' contract for professional development. The SIST is making the 
Waterbury public schools more responsive to prevailing conditions, trends, 
arid changes related to student performance and staff development. SIST 
received an overall 60% performance rating from the district's teachers and 
administrators, with strong agreement among teachers, administrators, and 
SIST consultants about the strengths of the initiative and its startup 
problems. The appendix contains the principals', assistant principals', and 
teachers' surveys. (Contains 28 figures.) (SLD) 
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Waterbury Public Schools 
and 

Connecticut Academy for Education 

School Improvement Support Team 
SIST 

EXTERNAL EVALUATION STUDY 
JUNE 30, 1999 

Executive Summary 

INTRODUCTION 

In fell 1998, the Connecticut Academy for Education retained Curriculum Research & 
Evaluation (CRE) for the purpose of conducting an external evaluation of the School 
Inq)rovement Support Team (SIST), \\iiich the Connecticut Academy mq)lemented in 
Waterbury Public Schools, during the 1998-99 school year. The main goal of SIST is to 
provide a con:q)rehensive, cohesive set of programs and activities for inq)roving student 
performance in Waterbiuy Public Schools. The Connecticut Academy was a good choice for 
this contract because of its excellent reputation statewide for systemic reform of education 
and because it had provided assistance with curriculum development to Waterbury Public 
Schools during the previous three years. 

The Connecticut Academy’s SIST team began work with building princ^als on December 
5, 1998. School-based work began in mid- January 1999. The Connecticut Academy’s total 
contract cost for SIST — ^including fees for personnel, activity, bus transportation, and 
substitute teacher reimbursement — ^is $152,309.02. 

More specifically, SIST’s educational consultants would collaborate with administrators and 
teachers in seven identified elementary schools (Pre-K to 5 and K-5); align work to and be 
consistent with the work done during the 1997-98 school year by the Connecticut Academy; 
review identified schools’ instructional programs measured by existing student performance 
data and give special attention to making the district’s instructional practices, assessment 
across grade levels, and identification of resource allocation practices at the elementary 
schools consistent with and supportive of the state curriculum frameworks; and work with 
the Waterbmy Public Schools administration to analyze stafBng expertise and student mobility 
issues. SIST’s specific areas for focus are assessment, mathematics, and language arts. Thus, 
SIST wouldprovide strategic, data based technical assistance to teachers, in order to in^rove 
students’ performance with academic skills, especially mathematics and language arts, as 
measured by the Connecticut Mastery Test (CMT). 
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Until recently, Connecticut’ s school administrators did not have the opportunities for strategic 
planning and dedsion-making that are now available to them. Historical analysis of 1993 to 
1 998 CMT student performance data is a powerful new tool for school administration. Hence, 
SIST offers Waterbury Public Schools administration a new and an innovative approach to 
in^roving key program areas, especially in curriculum development, assessment, and 
professional development. The district’s recent decision to include seven additional one-half 
days in the teachers’ contract for professional development, indicates the School Board’s 
commitment to in^roving the educational program and provides the opportunity for 
continuing this initiative. 

CRE’s analysis of CMT data clearly reveals not only that significant numbers of fourth grade 
students in the seven identified elementary schools perform below state goal on mathematics 
and language arts, but it also displays \\hat are the specific objectives on which successive 
cohorts of fourth grade students &il to meet state goal This data based decision-making 
process is continuously and directly concerned with students’ academic performance. The 
identification and display of specific deficiencies enables central office administrators and 
building princ^als to make changes quickly and decisively — ^to increase or decrease budget 
hems, to purchase materials and equ^ment, to revise their ciuiiculum, to adopt performance 
assessment instruments, and to design professional development. As a result, the Waterbury 
Public School district and hs individual buildings are more responswe to prevailing condhions, 
trends, and changes regarding student performance and staff development needs. 
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STUDENT PERFORMANCE DATA 

CRE’s analysis of student performance data — erq)hasizing mathematics, with a reduced focus 
on language arts — indicates that the state mandated CMT effectively identifies skill areas 
where cohorts of students tend to score low and, consequently, where teachers would most 
likely benefit fi'om strategic technical assistance. Thus, a strategic, data based decision-making 
process for curriculum development, assessment, and professional development — ^like that 
envisioned by SIST — can be of substantial relief and assistance to a school district, like 
Waterbuiy. 

CRE’s analysis of recent building-level CMT data shows that Wilson Elementary School has 
65.6 percent of its fourth grade students performing below the state’s proficient level in 
mathematics. Barnard, Chase, and Dtiggs elementary schools have approximately 30% more 
of their students than elsewhere in the district, who will need intervention for inq>roving their 
mathematics achievement. Bucks Mil, Hopeville, and Walsh elementary schools conq)are 
fevorably with the average student performance rating for the district, but this average 
performance level is lower than the state average. Thus, depending on which of the schools 
is enq)hasized, fi'om 20 to nearly 60 percent of thel 998-99 fourth grade students need 
intervention, in order to iiiq)rove their mathematics achievement for the sixth grade CMT. 

Analysis of CMT performance records, fi’om 1993 to 1998, for the fourth grade at the seven 
identified elementary schools ^ows that buildings have their own peculiar histories of low 
student perfo rman ce with specific mathematical skills. CRE strongly recommends that the 
Coimecticut Acadenty and the W aterbuiy Public Schools provide strategic technical assistance 
to K-5 teachers on the mathematical skills in need of attention that are peculiar to their 
building. This report includes lists of ^ecific mathematical skills that merit attention, 
organized by school building. 

Also, analysis of data reveals that all seven schools share a common set of specific skills on 
which many students consistently score below state goal Thus, CRE strongly recommends 
extensive, strategic, and consistent technical assistance to K-5 teachers, who work in all seven 
elementary schools, in these following eleven mathematical ^dll areas for curriculum 
developmen t, for performance-based assessment, and for professional develop ment. 

1. Extend and complete patterns 

3. Identify alternate forms using regrouping 

5. Relate multiplication and division facts to arrays and pictures 

13. Solve problems: estimate sum and difference, including money 

17. Create bar graphs and pictographs from data 

18. Write story problems from number sentences 

21. Solve story problem using extraneous information (5 of 7 schools) 

23. Identify needed information in problem situations (6 of 7 schools) 

26. Estimate laigth and area (6 of 7 schools) 

27. Estimate length and area 

28. Tell time (3 of 7 schools) 
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Also, a substantial number of the 1998-99 fourth grade students in these seven schools will 
most likely need assistance or intervention with development of these skills prior to taking the 
Mxth grade version of the CMT in fell 2000. 
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IMPLEMENTATION OF SIST 

SIST was inq)lemented in each of the seven identified schools and, despite encountering 
significant obstacles and a nuniber of setbacks, it nevertheless realized a measure of success. 
Additionally, wlien acknowledging that SIST is a truly innovative, a timely, and a useful 
approach to mq)roving public education — especially in urban communities, the lessons learned 
in the 1998-99 school year present the Waterbury Public Schools and the Connecticut 
Academy with e?q>licit guidelines for effective follow-up action. Thus, with the refinement of 
SIST’s design and operations, the school district is now in a very good position to benefit 
from its continued technical assistance in the 1999-2000 school year. 

Analysis of data from interviews and surveys shows that the irq)lementation of SIST proved 
to be a constructive process of discovery for everyone \\ho was directly involved. 
Representatives from the Connecticut Academy discovered that the long wait from August 
1998 to January 1999 for a contract from the City of Waterbury led to troublesome delays 
with SIST’s development and inqilementation. As a result, the time for planning, organization, 
schedule, and consulting services was more con^ressed than anticipated. The public 
announcement in the local newspaper of SIST’s operations in the seven schools caught 
princ^als and teachers by surprise and had negative repercussions for consultants. The 
shortage of substitute teaches, who were required for SIST’s operations, frustrated teachers, 
principals, and consultants. Finally, everyone associated with SIST acknowledgesthat student 
performance and teachers’ professional development are district-wide concerns, but 
mq)lementing revealed to all that the design and inq)lementation of a strategic technical 
support team must be based upon social and educational conditions at the building level. 

Teachers in two schools (namely Barnard and Bucks Hdl) assessed SIST’s performance at 
high levels, indicating that the mathematics consultants, in particular, provided workshops 
that were highly interesting and contained useful materials and methods. Additionally, 
princq)als said that, due — at least in part — to SIST’s conq)licated introduction, a majority of 
teachers in all seven schools now understand and value the concept of strategic, data based 
decision-making for curriculum, assessment, and professional development. 

Nevertheless, teachers at the other five schools (Chase, Driggs, Hopeville, Walsh, and 
Wilson) assessed SIST’s overall performance at satis&ctory and below average levels. 
Princq)als’ assessment of SIST complimented the teachers’ assessment on both general and 
specific items. Also, principals gave SIST an overall rating of six on a ten-point scale — above 
average, while stressing that they expected to report an eight or nine — exceptional. 

Additionally, teachers and administrators from all seven schools formd the language arts 
conponent lacking in regard to content and delivery. Also, the educators said that the 
assessment component friled from the start and was discontinued by SIST, which raised 
questions about the initiative’s related components in mathematics and language 

arts. Other concerns raised by the professional educators were that SIST devoted too little 
time to advanced planning with princpals and teachers; gave too little emphasis to 
programming for grades K-3; and used fa miliar or out-of-date resources too frequently. 
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LESSONS LEARNED 
The main lessons learned in this initiative are: 

• the individual school building, staf^ students, and piinc^al nmst constitute the focal 
unit for SIST’s planning, design, scheduling, operations, personnel, and evaluation; 

• p lanning must begin well in advance of the first day of school — during the preceding 
summer or ^ring of previous school year; 

• principals and lead teachers must be directly involved in SIST’s planning, 
development, and — ^when feasible — ^its operations, in order to promote collaboration 
at all levels in the school’s organization, mutual ownersh^, and accountability; 

• public announcements about the initiative should occur after staff and administration 
have been alerted to the decision to include SIST in a buildmg’s work schedule; 

• consultants must have strengths in areas of critical need, including e 7 q>ertise with 
teaching in ^ecific grade levels and with academic skills, and they must provide 
worktops and other services that are valued highly by the teachers and princ^als; 

• the School Board, central office, teachers, principals, consultants, and the Connecticut 
Academy must see SIST as a long-term, intensive, thoughtful, and process-oriented 
approach to school improvement; 

• patience, trust, consistency and coherence, accoimtability, and determination to 
succeed are critically important values to assure short-term and long-term 
success — trust between teachers, princpals, and consultants is essential 
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CONCLUSION 

CRE’s evaluation study of the SIST initiative in the seven identified elementary schools of the 
Waterbury Puhhc Schools demonstrates that a data based approach to decisiommaking is a 
highly useful, strategic approach to educational programming. (CRE encourages the audience 
to read the fid! report on the evaluation of SIST.) It is a procedure that Wateibury’s school 
administrators and teachers prefer, rather than continuing obsolete methods. Also, the first- 
year implementation of a data based decision-making process — ^like any other inq)ortant and 
11 R efill innovation — entails con:q)Hcations, which lead to setbacks that necessitate revisions. 
Hence, this was a discovery process for all \^dio were directly involved. Additionally, SIST 
is a thoughtful approach to program development, requiring time and consistent appUcation, 
in order to realize success. The analysis of data shows mixed resuhs overall, which should not 
surprise or disappoint the reader. 

Briefly, analysis of data obtained fi’om surveys with teachers, structured interviews with 
administrators and representatives fi’om the Connecticut Academy, observations ofmeetings, 
and review of documents shows that SIST produced mixed results in the schools. The 
evaluation shows strong agreement between teachers, administrators, and SIST consultants 
re garding the Strengths of this initiative, its start-up problems, and the best solutions to those 
problems. 

From Waterbury’s teachers and administrators, SIST received an overall 60% performance 
rating. Similarly, consultants fi’om the Connecticut Acadenty ejq>ressed disappointment that 
the program had not gone as smoothly as they had expected. However, the SIST idea for 
providing technical assistance to elementary schools is recognized by all the educators as a 
very sotmd approach to mq)roving students’ learning, in general, and for inq)roving their 
performance on the CMT, in particular. Despite the issues regarding SIST’s inqjlementation, 
it is nonetheless a program that will benefit the district. Thus, the assistant superintendent 
should be commended for his foresight and risk-taking. 

Educational innovations always encotmter resistance fi’om various sources, contend with 
various obstacles, and have to regroup or to reorganize for a more effective approach to the 
situation — ei^ecially in their first year of operations. Thus, recognizing the strengths of this 
idea for inq)roving the instructional program and its general plan of operations, CRE strongly 
recommends that both parties to this contract exploit A^at works best and continue working 
together to inq)rove teaching and learning in Waterbury’s elementary schools. At the 
conclusion of this report, there are specific recommendations for inq)rovement. 

At this time, the Connecticut Academy and the Waterbury PubHc Schools should collaborate 
to clarify SIST’s purpose and to estabhsh rigorous and specific objectives, processes, and 
standards for the initiative. In summer 1 999, a year-long technical assistance program should 
resume in two or three of the seven original elementary schools. By fall 1999, the revised 
SIST program should attain higher approval ratings fi’om teachers and princ^als. It should 
also provide greater assurance of success for teachers and students. Once the Waterbury 
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model of technical assistance has achieved definition and has demonstrated its effectiveness 
in these schools, it should be applied with appropriate adjustments on a scheduled basis in 
other elementary schools in Wateibury. 

In conclusion, the lessons learned are the best results of this initiative. Patience, mutual trust, 
consistency, coherence, accountability, and determination to succeed must continue as the 
basic principles guiding SIST’s development. The Connecticut Academy’s reputation for 
providing quality service and materials to improve mathematics and science education is well- 
recognized statewide. Administrators and teachers in the Waterbury Public Schools e7q>ress 
a commitment to inq>roving students’ CMT performance through the strategic professional 
development of elementary school teachers. Furthermore, everyone involved with this 
initiative recognizes that strategic planning and operations — ^informed by a current and e;q)licit 
anafysis of essential data — are critical elements for inq)roving the school’s curriculum, 
assessment practices, and professional development. With these recommendations in mmd, 
CRE strongly urges continiiation of SIST in the Waterbury Public Schools. 
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RECOMMENDATIONS 

The following recommendations come from surveys and interviews with Waterbury’ s teachers 
and princ^als and from reports and meetings with the consultants, \\fro worked for the 
Connecticut Academy. Additionally, CKE provides a number of recommendations for analy^ 
of CMT data and SIST program changes, in order to support an effective technical assistance 
program. 

1. Anafysis of CMT Data 

• Analyze CMT data by cohort groups of students within buildings and, to the extent 
possible, within the district, in order to study the long-term intact of intervention 
strategies and to assess the merit of different intervention strategies. 

• Hire a consultant — ^\\fro is familiar with CMT data, urban education issues, 
curriculum, instruction, and assessment — ^to develop graphical representations and 
related materials pertaining to building-level student performance data for use by 
teachers and principals when planning for technical assistance. 

• Focus on the K-5 students wbo do not change schools, using as the mmimum criterion 
of success for the SIST program 100% of these students scoring at or above state 
goal. Develop a similar plan for aU other children, based on recommendations from 
the study of student mobility issues in Waterbury. 

• Incorporate appropriate performance-based assessment instruments to monitor all 
children’s development of speciSc skills in grades K-5, including relevant follow-up 
methods and materials for developmental and remedial instruction. 

• Use the yearly analysis of CMT data (as suggested in this report) to identify specific 
skills for special age/grade appropriate en^)hasis in each grade K-5, using 85 percent 
mastery (state goal) on all performance measures as the main criterion of success. 

2. SIST Program 

• Maintain regular communications with central office administrators in regard to 
SIST’s operations, in general, and to feedback from participants, specifically. 

• En:q>hasize \^diat works. Demonstrate to teachers the methods and materials that help 
urban children to learn reading, writing, and/or mathematics. 

• Organize and frciUtate a sufficient number of discussion sessions, during which 
teachers describe common issues and propose effective solutions. 

• Monitor on a daily basis the activities and processes of consultants, in order to ensure 
quality performance, including pronq>t response to issues and problems. Set the goal 
for each building at 1 00% participant satisfrction that SIST provides excellent service. 
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• Consult with teachers and principals at the earliest opportunity to determine budding 
e7q>ectations and needs, and continue consulting on a regular basis throughout the 
initiative. 

• Focus on intensive, strategic, age/grade appropriate developmental programs in math 
and reading/study skills for Pre-K through third grade. Set 85 percent mastery of 
^ecific skills (state goal) as the criterion of success. 

• Provide follow-up developmental and remedial programs in math and reading/study 
skills for fourth and Mh grade students. Set 85 percent mastery of specific skills 
(state goal) as the criterion of success. Monitor closely for age-appropriate 
development and adjust instruction based on students’ needs. 

• Transition district administrators, piinc^als, and lead teachers within three years to 
fiill responsibility for analysis of CMT data and related documentation, in order to 
support data based decision-making at district and building levels. 

• Hire consuhants to study and report on student mobility issues in Waterbury as they 
relate to curriculum, instruction, and assessment. Develop recommendations for 
policy changes, general program development, and intervention strategies in order to 
mitigate the adverse effects of extremely high student mobility rates. 

3. Personnel 

• During 1999 summer, hire a consultant to study CMT data and to work with district 
administrators, principals, and lead teachers to identify specific skills as objectives for 
SIST. Select methods, materials, equipment, and personnel to acconqplish the 
objectives. 

• Hire exenq)lary technical assistance providers, with qualifications — ^including practical 
e?q)erience — ^in pre-kindergarten and kindergarten and in grades 1, 2, 3, 4, and 5. 

• Ifire a consultant for the assessment conqponent, A\iio has substantial, direct 
e7q>eiience with anafysis of CMT data for data based decision- making , and A\iio has 
the skill to work effectively with teachers and administrators. 

• Insist on high quality performance by all consultants and ensure fit of personnel and 
program with individ\ial schools. 

• Inq)rove availability and scheduling of substitute teachers. 

4. Operations 

• Conduct intensive study of CMT data and other relevant documentation, in order 

to prepare a clear, formal, consistent, and data based plan for professional 
development that is specific to each building. 
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• Disseminate the technical assistance plan to the teachers and princ^al, in order to 
^pport direct, open, and continuous communication with all pardc^ants. 

• Provide sustained, consistent, strategic technical assistance (have princ^al and 
lead teachers he^ determine the time frame and schedule of operations). 

• Organize consultants’ work to include presenting workshops that are consistent 
with the plan, collaborating with teachers, working hand-in-hand with regular 
classroom teachers, serving as grade level resource teachers, and identifying lead 
teachers to assist with inoplementing the program. 

• Organize workshops by age/grade levels, because the needs and interests of pre- 
kindergarten, kindergarten, primary school, and intermediate level teachers are 
very different. 

5. Evaluation and Assessment 

• Continue the independent external evaluation of the SIST initiative, in order to 
describe, measure, and assess its performance during this formative period. 
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Waterbury Public Schools 
and 

Connecticut Academy for Education 

School Improvement Support Team 
SIST 

EXTERNAL EVALUATION STUDY 
JUNE 30, 1999 

INTRODUCTION 

In thel 998-99 school year, the Waterbiuy Pubhc Schools — ^through the City of Waterbury, 
CT — and the Connecticut Academy for Education in Mathematics, Science, and T echnology * 
adopted an agreement specifying that the Connecticut Academy for Education would develop 
and hi 5 )lement a conq>rehensive, cohesive set of programs and activities to inq)rove delivery 
of instruction for inq)roved student performance in the Waterbiuy Pubhc Schools. School 
Inq)rovement Team Support (SIST) is the name that was adopted for this program. The 
Connecticut Academy was a good choice for this contract because of its excellent reputation 
statewide for systemic reform of education and because it had provided assistance to the 
school district on curriculum development during the previous three years. 

The Connecticut Academy’s total contract cost for SIST — ^including fees for personnel, 
activity, bus transportation, and substitute teacher reimbursement — ^is $152,309.02. Formal 
discussion of this technical assistance program was initiated by representatives from the 
Waterbury Pubhc Schools and the Connecticut Academy in August 1998. The Connecticut 
Academy’s SIST team began work with building principals on December 5, 1998. School- 
based work began in nhd- January 1999. 

SCHOOL IMPROVEMENT SUPPORT TEAM (SIST) 

1. Purpose 

As indicated above, the overafl purpose of SIST is to improve delivery of instruction, leading 
to improved student performance. More specifically, SIST wouldprovide technical assistance 
to the Waterbury Pubhc Schools — ahgned to and consistent with the work done during the 
1997-98 school year by the Connecticut Academy — and make the district’s educational 



‘Connecticut Academy for Education, Connecticut Academy, and Academy are common 
references for the Connecticut Academy for Education in Mathematics, Science, and 
Technology. 
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program consistent with the Curriculum Frameworks for mathematics and language arts, 
w^ch have been approved for statewide in:q)lementation by the Connecticut State Department 
of Education. Additionally, SIST would review identified schools’ instructional programs 
measured by existing student performance data and give special attention to making the 
district’s instructionalpractices, assessment across grade levels, and identification of resource 
allocation practices at seven identified elementary schools consistent with and supportive of 
the Curriculum Frameworks. Also, SIST would work with the Waterbxuy Public Schools 
administration to analyze staffing expertise and student mobility issues. 

Diuing the 1998-99 school year, the following seven Pre-K-5 and K-5 schools partic^ated 
in SIST. Included in this list is information regarding student enrollment, student mobility 
rates^, andniunber of regular classroom teachers. The extent offeculty mobility is not an issue 
in these seven schools. 



Schools 


Enrollment 


Mobility Rate 


Teachers 


Barnard 


285 


100% 


18 


Bucks Hill 


580 


32% 


27 


Chase 


747 


77% 


33 


Driggs 


598 


100% 


33 


Hopeville 


480 


34% 


24 


Walsh 


615 


100% 


27 


Wilson 


270 


very low 


12 



2. Expected Results 

The contract between Waterbxxry Public Schools and the Connecticut Academy specifies the 
following expected results: 

• analyas of conq>rehensiveness and coherence of mathematics and language arts 
curricula and instructional practices 

• group and individual work with the seven Waterbury school faculty for a coherent 
program of instruction consistent with the Curriculxmi Frameworks 

• co ntinuin g dialogue between SIST and the seven building ac c mistrators relevant to 
technical assistance set forth in this plan 

• analysis of &culty professional development practices for strategic planning 



^aterbxxry’s student mobility rate involves several categories, including transfers within and 
outade of the district. To sin^lify the issue, we use one figiue — an approxim' .tion drawn 
fi’om the princ^al’s office that combines all terms. Additionally, these percent >i'?s do not 
indicate that, for exan^le, 1 00% of the students actxiaQy transferred, but rather that a number 
of instances eqxial to the total number of students enrolled for the year made transfers. 



Curriculum Research & Evaluation^ 237 Singletcn Road, Ch^lin, CT 06235^860-455-l229^Fax 860-455-001 1 ♦www.creu&ocjm 



1999 SIST Evahition: Waterbuiy Page 3 



• analysis of data on &culty mobility and school performance 

• report of results by hme 30, 1999. 

Additional^, the tedmical assistance contract includes measurable objectives for the following 
areas: 

• Teaching and Learning Goals and Objectives 

• Curriculum and Instruction 

• Assessment 

• Professional Development 
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FOCUS OF STUDY 

The Connecticut Academy retained Ciuriculiun Research & Evaluation as evaluator for the 
purpose of conducting a formative evaluation of SIST’s operations, during the 1998-99 
school year. The process of evaluation included: participation in SIST’s organizational 
meetings at the Connecticut Acadeny; participation in SIST’s preliminary meeting at the 
Waterbury PubKc Schools; development and distribution of survey instruments for teachers; 
development of interview questions and conduct of interviews with adioinistrators and 
representatives from the Connecticut Academy; collection and analysis of data from various 
sources, including surveys, interviews, and results of students’ performance on the 
mathematics and language arts portions of the CMT ; and preparation of the formative report. 

The design fi)r .this evaluation study used procedures and instruments for data collection and 
analysis that are consistent with national standards for evaluation studies. Additionally, key 
elements in the successful assessment of a process-oriented program, such as the SIST 
initiative in Waterbury, are flexibility and caution. Consequently, the evaluator developed 
protocol for the field study in reference to and contingent upon SIST’s development in the 
different elementary schools. 

Similarly, the evaluator’s development of data collection instruments relied on clarification 
of SIST’s program and the response of participating teachers and princ^als. Thus, in order 
to lesson the impact of the evaluation process on district personnel and on the educational 
program, the evaluator — ^in consultation with the Connecticut Academy — did not observe 
workshops and teachers’ classrooms. Additionally, the evaluator did not administer pre- and 
post tests, which would indicate the extent to which SIST influenced teachers’ practice and 
opinions over time. Also, the items on the teacher survey were limited, in order to encourage 
all teachers to respond. 

Thus, a weakness of the evaluation is the absence of experimental control in the research 
deidgn. Nevertheless, CRE maintains that the evaluation design is appropriate to this situation. 
Additionally, CRE maintains that this report of resuhs is accurate and truthful. CRE sincerely 
hopes that this evaluation is useful to the Waterbury Public Schools and to the Connecticut 
Academy for Education. Contributions to the evaluation process by teachers, administrators, 
and representatives from the Connecticut Academy are greatly appreciated. Assessment 
instruments developed for this study are found in the Appendix. 
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PREVAILING ISSUES 

Analysis of data obtained from interviews with key participants and surveys from teachers 
revealed that a number of issues had a significant impact on the development and 
inq>lementation of SIST. The Connecticut Academy reported that the long delay (from 
August 1998 to January 1999) in finalizing the contract with the City of Waterbury 
conq)ressed the time that was anticipated for planning and implementing the program 
Nevertheless, on the basis of assurances from the assistant superintendent that the contract 
would be approved, the Connecticut Academy initiated the SIST program Teachers and 
principals were suiprised to learn from the local newspaper that this initiative was coming to 
their buildings. They preferred to be involved in the planning stage of SIST and, thus, not only 
to have advance notice, but also to have a role in determining its design and operations. 
Indeed, some of the teachers and administrators could collaborate with the consultants to 
provide technical askance. 

Additionally, due to poor communication, teachers became concerned that the initiative was 
actually serving an evaluative role relative to their classroom work, instead of focused on 
technical assistance to improve teaching and learning. Also, teachers, administrators, 
consuhants, and representatives from the Connecticut Academy said that the shortage or the 
absence of substitute teachers often delayed or canceled operations that were planned, which 
caused many partic^ants to feel finstrated because things were not done as expected. The 
Connecticut Academy acknowledged that the failure of the assessment and language arts 
conq)onents led to some disappointment with the program In the future, the Connecticut 
Academy plans to research more carefully the needs and the preferences of individual 
buildings, in order to select consuhants that provide the professional educators with the best 
technical assistance. 
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ANALYSIS OF STUDENT PERFORMANCE DATA: MATHEMATICS 
In 1995, CRE created a graphical system for the Connecticut Academy in order to represent 
statewide student performance data from the Connecticut Mastery Test (CMT) (2“* 
Generation). The Connecticut State Department of Education (CSDE) administers the CMT 
annually in grades four, six, and eight. There are three sections, including mathematics, 
reading, and writing. The graphical system presents in visual form a longitudinal study of 
students’ performance on the CMT at both district and building levels, including analysis of 
specific skills, thus, enabling the reader to quickly recognize positive and negative trends. The 
graphical system was used for representation of data from the mathematical conq>onent of the 
CMT, however, the same system may be applied to the language arts conq>onent. 

The current version of the graphical analysis of student performance data was produced by 
the Connecticut Academy, in spring 1 999, and includes documentation from 1 993 - 1 998. For 
the purposes ofthis discussion, the reader is advised to consult the binder containing “Student 
Performance in Mathematics, Connecticut Mastery Test 1993 - 1998: A Graphical 
Presentation Prepared for Waterbiuy Public Schools.” There will be a discussion of graphs 
contained in Part I that is focused on analysis of fourth grade students’ overall performance 
specific to the seven elementary schools that were identified for SIST. Next, there will be a 
discussion of graphs contained in Part II that concerns analysis of fourth grade students’ 
performance on specific skills within the seven elementary schools. 

CSDE categorizes student performance data from the CMT for grades four, six, and eight 
according to three levels: Intervention, Proficient, and Excellent. For this analysis of data, 
CRE is restricting its focus to the intervention level for the district’s fourth grade students at 
the seven identified elementary schools. Also, the CSDE has set maximum scores and other 
criteria for the grades, >^ch appear in the following table. The last column in this table shows 
the percent of students statewide who scored at or above the state goal in all three sections 
of the CMT in fall 1 998. Additionally, analysis of CMT data by the CSDE reveals that, from 
1993 to 1998, there has been an 8.1% overall increase for fomth graders, an 8.0% increase 
for sbcth graders, and a 10.5% increase overall for eighth graders statewide scoring at or 
above goal on all three sections of the test combined.^ 





Total 

Points 


State Goal 


Intervention 


Proficient 


Excellent 


% At/Above 
Goal 


Grade 4 


121 


103 (85%) 


<77 


77-102 


103 + 


61.4% 


Grade 6 


154 


122 (79%) 


<80 


80-121 


122 + 


52.9% 


Grade 8 


172 


130 (76%) 


<84 


84-129 


130 + 


56.7% 



^For more information on CSDE documentation and analysis of CMT data, the reader should 
consult the CSDE web site at http://www.state.ct.us/sde or call the CSDE’s Public 
Information OflEice at 860-566-5677. 
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1. Analysis of Students* Overall Petfomtance 

Figure 17 presents overall student performance data for fourth grade students, who scored 
at the intervention level on the mathematics section of the CMT for fell 1998 — categorized 
by state, Waterbury Public Schools (district), and seven identified elementary schools in 
Waterbury, CT. Clearly, Woodrow Wilson Elementary School has the most serious condition, 
with 65.6 percent of its fourth grade students performing below the state’s proficient level in 
mathematics. Results for Barnard, Chase, and Diiggs ^ow approximately 30% more of their 
students than else\^fiere in the district, who will need intervention for improving their 
mathematics achievement. Bucks (fill, Hopeville, and Walsh conq)are fevorably with the 
average student performance rating for the district, A^ch is still substantially different from 
and lower than the state average. 

Hence, across the seven identified schools, there are three levels of performance: average, 
below average, and fiir below average for the district. When this district- and bmlding-level 
data is compared with the statewide performance relative to intervention needs (10%), it is 
clear that from 20 to nearly 60 percent ofWaterbury’s 1998 fourth grade students may need 
intervention in order to inprove their mathematics achievement. 

Thus, this analysis of fourth grade student performance data indicates that — ^based on an 
average class size of 20 students (approximately the district and state average) and depending 
on the school — ^between six and twelve students per class will need intervention, in the 1 999 - 
2000 school year, in order to inprove their chances of performing at least at the proficient 
level with mathematics achievement by the time they take the sbcth grade veraon of the CMT, 
infeU2000. 



Figure 1: 4th Grade Intervention 
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1998 



‘‘Wilson opened its doors for the first time, in 1 996-97, as a dual language school of choice, 
with Spanish & English required. 
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Analysis of fourth grade student performance data at each of the seven schools for years 1993 
through 1998 on the mathematics section of the CM! (see binder), reveals modest variation 
in the percentage of students needing intervention for mathematics achievement. The 
perfo rman ce of different cohorts of previous fourth grade students at Bucks Hill, D rigg s. 
Hopeville, and W alsh elementary schools indicates that increasing percentages of each 
succeedin g cohort &ired somew^iat better than their predecessors in regard to need for 
intervention. However, at Barnard, Chase, and Wilson the succeeding cohorts either had 
roughly similar performance results or the percentages needing intervention grew larger. 
Today, these many of these students are in the fifih, sbcth, and seventh grades. 

Finally, Figure 18 shows that analysis of district-level data — ^taking the 1993 performance 
record as baseline data and using 1998 data as the current reference point — on the fourth 
grade students’ performance on the mathematics section of the CM! indicates that there has 
been a modest 4.4% reduction in the percentage of students needing intervention, i^ch 
con^ares &vorably with the stateAvide reduction for these same data points. 

Figure 2: Reduction in Intervention 

5 
4 
3 
2 
1 




1993-98 
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Although the district’s 4.4% reduction for fourth grade students needing intervention in 
mathematics conq)ares fevorably with a 4.5% reduction statewide, Rgure 19 reveals that the 
average percentage of fourdi grade students statewide needing intervention between the years 
1993 to 1998 is 12.1%, wWle the average percentage of students needing intervention in 
Waterbury Public Schools during that same period is 34.1% — ^v^Wch is a dififer^ce of 22 
percentage points. 




M 1993-98 
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2. Analysis of Students* Specific Skills 

The graphical charts published by the Connecticut Academy for representation of specific 
skills is focused on the percent of students “at or above state goal” on each CMT grade four 
mathematics objective, A^ch is consistent with the CSDE annual report concerning CMT 
results. However, neither the CSDE data sets nor the charts exhibit the percent of students 
who would benefit most from intervention strategies, in order to mq)rove their mathematics 
achievement prior to the CMT’s administration in the sixth grade. 

The state goal for the CMT fourth grade objectives is 85% of the total score (state goal: 1 03 ; 
total: 121). Thus, “at or above state goal” is one percentage point above proficient, in addition 
to being the lowest percent acceptable in the “excellent” category.’ Since these students’ 
mathematical skills have been assessed as “excellent,” it is not necessary to focus on them for 
analysis and strategic decision-making when there are concerns for curricular issues and 
professional development. Where students’ performance registers low in these charts, it is at 
least clear that a great majority of students lack that specific mathematical skill 

For pruposes of this analysis, CRE will use interpolation of the charts from the Connecticut 
Academy concerning percent of fourth grade students “at or above state goals”between 1 993 
and 1998, in order to identify the specific mathematical objectives that are most obviously in 
need of attention in each of the seven different elementary schools. The analysis will also use 
interpolation of the charts to focus on specific skills that are most in need of attention across 
all of these schools. 

The criterion CRE will use to determine a building-level need for intervention with particular 
objectives will be identification of circiunstances where there is a mean or average percent 
lowing that “50 percent or more” of the fourth grade students did not perform “at or above 
state goal” on that objective, between 1993 and 1998. Additionally, we will consider strong 
trends in the data over three or more years (increase, decrease, or plateau) to merit special 
attention. 



’See the table for CSDE scoring criteria in the section above titled, “Analysis of Student 
Performance Data.” 
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The following table shows the categories that CSDE uses for its CMT objectives on general 
and specific levels. 



Objective Categmies 


Specific Skills 


Number Sense (1-5) 


• Extend and complete patterns 

• Identify alternate forms using expanded notation 

• Identify alternate forms using regrouping 

• Identify and label parts of regions and sets 

• Relate multiplication and division facts to arr^ and 
pictures 


Number Facts/Computation (6-9) 


• Add and subtract facts to 1 8 

• Add and subtract 1- and 2-digit numbers without 
regrouping 

• Add 1- and 2-digit numbers with regrouping 

• Multiply and divide by 2, 5, and 10 


Problem Solving/Aj^lication (10-24) 


• Solve problems: order and magnitude of wliole numbers 

• Solve problems: place value concepts 

• Solve problems: rounding 2-digit w^ole numbers 

• Solve problems: estimate sum and difference, including 
money 

• Identify objects and numbers that are same and different 

• Identify information from gr^h, table, and chart 

• Draw conclusions from graph, table, and chart 

• Create bar graphs and pictogr^hs from data 

• Write story pr^lems from number sentences 

• Identify operation to solve story problem 

• Solve story problem using addition and subtraction 

• Solve story problem using extraneous information 

• Solve problem concerning time, el^)sed time, and 
calendars 

• Identify needed information in problem situations 

• Solve problems with elementary notions of probability 


Measurement/Geometiy (25-30) 


• Measure lengths and draw lines (in centimeters) 

• Identify customary and metric measures 

• Estimate length and area 

• Tell time 

• Determine the value of a set of coins 

• Identify and draw geometric shapes and figures 
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The operational question for the following discussion ofbuilding-level results is: Based upon 
CMT data, which specific skills do 50% or more of the building’s fourth grade students 
perform below CSDE’s designation of "at or above state goal” (i.e., 85% mastery level)? 

The following table identifies the grades that these fourth grade students will be attending in 
the 1999 to 2000 academic year. 



Year of CMT 


Grade in 1999 - 2000 


1993 


1 £ 


1994 


9 


1995 


8 


1996 


7 


1997 


6 


1998 


5 



The above table shows that all of these students are school-age; that the 1 997 cohort will take 
the sixth grade version of the CMT in fall 1999; that the 1995 cohort will take the eighth 
grade version of the CMT in fell 1999; and that the 1993 cohort group will take the 
Connecticut Academic Performance Test (CAPT), which is also state mandated and is 
administered by CSDE, in spring 2000. 



A. Barnard 

Figure 20 identifies the specific mathematical skills that fourth grade students — ^w^o took the 
CMT at Barnard Elementary School between 1 993 and 1 998 — consistently scored low on and 
the percent of students who will most likely need assistance or intervention with these skills. 



Figure 4: Skill & Percent Deficient 

27. Est length & area 
23. Id needed info 
18. Write story prob 
17. Create bar graphs, etc 
13. Solve prob: est sum & diff 
5. Relate mult & div facts 
3. Id alt forms using regrouping 
1. Ext & complete patterns 

i i i i i I I 

0 10203040506070 




Barnard: 1993-98 C^/^’ Math 4th Grade 
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AddidonaUy, analysis of the charts on the 50 percent or more of fourth grade students vsho 
did not score at or above state goal between 1993 and 1998 on the CMT reveals that Barnard 
is e?q>eriencing a serious and consistent downward trend in students’ performance with the 
following specific mathematical skills: 



13. Solve problems: estimate sum and difference, including mon^ 

17. Create bar graphs and pictographs from data 

18. Write stm^ problems from number sentences 
23. Idoitify needed information in problem situations 

27. Estimate length and area 



B. Bucks Hill 

Figure 2 1 identifies the specific mathematical skills that fourth grade students — ^\^o took the 
CMT at Bucks Hill Elementary School between 1 993 and 1 998 — consistently scored low on 
and the percent of students who will most likely need assistance or intervention with these 
skills. 



Figure 5: Skill & Percent Deficient 



27. Est length & area 
26. Id customary & metric 
23. Id needed info 
21. Solve story prob 
18. Write story prob 
17. Create bar graphs, etc 
13. Solve prob: est sum & diff 
5. Relate mult & div facts 
3. Id alt forms using regrouping 
1. Ext & complete patterns 




H Bucics Hill: 1993-98 CI^/TT Math 4th grade 



Additionally, analysis of the charts on the 50 percent or more of foxuth grade students who 
did not score at or above state goal between 1993 and 1998 on the CMT reveals that Bucks 
HSU is e?q>eriencing a consistent low performance with the following mathematical skill: 



^S^J^o^fyaltvnatefbrmsjisi^^ 
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There is evidence of a serious and consistent downward trend in students’ performance with 
the following specific mathematical skills: 



13. Solve problems: estimate sum and difference, including money 
27. Estimate length and area 



Finally, despite a generally low-level performance between 1993 and 1998, analysis of data 
nevertheless indicates that Bucks FGU is realizing substantial and consistent growth in the 
following skills : 



18. Write story problems from number saitmices 
21. Solve ^ry problem using extraneous information 
23. Identify needed information in problem gtuations 



C Chase 

Figure 22 identifies the specific mathematical skills that fourth grade students — ^wiio took the 
CMT at H. S. Chase Elementary School between 1993 and 1998 — consistently scored low 
on and the percent of students v\^o will most likely need assistance or intervention with these 



skills 



Rgure 6: Skill and Percent Deficient 

28. Tell lime 
27. Est length & area 
26. Id customary & metric 
23. Id needed info 
21 . Solve story prob 
18. Write story prob 
17. Create bar graphs, etc 
13. Solve prob: est sum & diff 
5. Relate mult & div facts 
3. Id alt forms using regrouping 
1 . Ext & complete patterns 



^ Chase 1993-1998 CMT Math 4th Grade 

Additionally, analysis of the charts on the 50 percent or more of fourth grade students w^o 
did not score at or above state goal between 1993 and 1998 on the CMT reveals that Chase 
is eiqieriencing a serious and consistent downward trend in students’ performance with the 
following ^ecific mathematical skill: 




0 1020304050607080 



28. TeU time 
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Finally, despite a generally low-level performance between 1993 and 1998, analysis of data 
nevertheless indicates that Chase is realizing substantial and consistent growth in the 
following skills: 



5. Relate multiplication and division facts to arrays and pictures 

17. Create bar graphs and pictographs fr<mi data 

18. Write story problems from number sentences 



D. Driggs 

Figure 23 identifies the specific mathematical skills that fourth grade students — \^o took the 
CMT at Driggs Elementary School between 1 993 and 1 998 — consistraitly scored low on and 
the percent of students v\ko will most likely need assistance or intervention with these skills. 

Figure 7: Skill and Percent Deficient 

27. Est length & area 
26. Id customary & metric 
18. Write story prob 
17. Create bar graphs, etc 
13. Solve prob: est sum & diff 
5. Relate mult & div facts 
3. Id alt forms using regrouping 
1 . Ext & complete patterns 

0 20 40 60 80 100 
^ Driggs: 1993-98 C^/r^ Math 4th Grade 

Analy^ of the charts on the 50 percent or more of fourth grade students who did not score 
at or above state goal between 1 993 and 1 998 on the CMT reveals that Driggs is e?q)eriencing 
a serious and consistent downward trend in students’ performance with the following i^ecific 
mathematical skills: 




3. Identify alternate forms using regrouping 
17. Create bar graphs and pictographs from data 
27. Estimate length and area 
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Additionally, analysis of the charts on the 50 percent or more of fourth grade students v^dio 
did not score at or above state goal between 1993 and 1998 on the CMT reveals that Driggs 
is e?q)eiiencing a conastent low performance with the following mathematical skill: 



3. Idoitify alternate forms uang regrouping 

13. Solve problems: estimate sum and difference, including money 

17. Create bar graphs and pictographs from data 



Finally, despite a generally low-level performance between 1993 and 1998, analysis of data 
nevertheless indicates that Ehiggs is realizing substantial and consistent growth in the 
following skills : 



5. Relate multiplicatimi and divisimi facts to arrays and pictures 
18. Write story problems from number sentences 



E. Hopeville 

Figure 24 identifies the specific mathematical skills that fourth grade students — ^who took the 
CMT at Hopeville Elementary School between 1993 and 1998 — consistently scored low on 
and the percent of students a\^o will most likely need assistance or intervention with these 
skills 



Figure 8: Skill and Percent Deficient 

27. Est length & area 
26. Id customary & metric 
24. Solve prob: elem probability 
23. Id needed info 
21 . Solve story prob 
18. Write story prob 
17. Create bar graphs, etc 
13. Solve prob: est sum & diff 
5. Relate mult & div facts 
3. Id alt forms using regrouping 
1. Ext & complete patterns 

0 1020304050607080 

^ Hopeville: 1993-98 CMT Math 4th Grade 

Analysis of the charts on the 50 percent or more of fourth grade students \^o did not score 
at or above state goal between 1993 and 1998 on the CMT reveals that Hopeville is 
experiencing a serious and consistent downward trend in students’ performance with the 
following specific mathematical skill : 

\ 3. Identify alternate forms using regrouping | 
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Additionalfy, analysis of the charts on the 50 percent or more of fourth grade students \\ho 
did not score at or above state goal between 1993 and 1998 on the CMT reveals that 
Hopeville is e?q)eriencing a consistent low performance with the following mathematical 
skills: 



3. Ittentify altemate fraims using regroiping 

23. Identic needed information in pr<rit>lem situations 



Fmalfy, despite a generally low-level performance between 1993 and 1998, analysis of data 
nevertheless indicates that Hopeville is realizing substantial and consistent growth in the 
following skOls: 



5. Relate multiplication and division facts to arrays and pictures 

17. Create bar graphs and pictographs from data 

18. Write story problems from number sentences 

23. Identify needed information in problem situations 
« Solve problems wifli dementary notions of pr<d)ability 



F. Walsh 

Figure 25 identifies the specific mathematical skills that fourth grade students — ^who took the 
CMT at Waldi Elementary School between 1993 and 1998 — consistently scored low on and 
the percent of students vsho will most likely need assistance or intervention with these skills. 



Rgure 9: Skill and Percent Deficient 

27. Est length & area 
26. Id customary & metric 
24. Solve prob elem probability 
23. Id needed info 
21. Solve story prob 
18. Write story prob 
17. Create bar graphs etc 
13. Solve prob: est sum & diff 
5. Relate mult & div fads 
3. Id alt forms using regrouping 
1. Ext & complete patterns 
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Walsh: 1993-98 C^4T Math 4th Grade 



Analysis of the charts on the 50 percent or more of fourth grade students vsdio did not score 
at or above state goal between 1993 and 1998 on the CMT reveals that Walsh is experiencing 
a serious and consistent doAvnward trend in students’ performance with the following ^ecific 
mathematical skills: 
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3. Identify alternate forms using regrouping 
27. Estimate length and area 



Additionally, analysis of the charts on the 50 percent or more of fourth grade students who 
did not score at or above state goal between 1 993 and 1998 on the CMT reveals that Walsh 
is e?q)eriencing a consistent low performance with the following mathematical skills: 



3. Idmtify alternate forms using regrouping 

13. S<dve problems: estimate sum and difference, including money 

17. Create bar graphs and pictographs from data 



Finally, despite a generally low-level performance between 1993 and 1998, analysis of data 
nevertheless indicates that Walsh is realizing substantial and consistent growth in the 
following skills : 



5. Relate multiplication and division facts to arrays and pictures 

18. Write story problems from number sentences 

21. Solve story problem using ertraneous informaticm 

24. Solve problems with elementary notions of probability 



G. Wilson 

There are only two years of CMT data from Wilson — 1997 and 1998, thus, the following 
analysis of fourth grade students’ performance with the mathematics section of the CMT is 
seriously limited. Nonetheless, there is sufficient evidence indicating what are most likely the 
main problem areas. In fret, analysis of data indicates that, in conq)arison with the other six 
identified schools, Wilson has the greatest munber of mathematical skills on vriiich students 
are deficient. 



Figure 10: Skill and Percent Deficient 



30 Id & efraw geo shapee 
2B. Tdltbne 
27. Eat length & area 
26 l Id cuatomary & metric 
23. Id needed info 
22. Solve prob: time, etc 
2CX Solve atofy prob 
Id. Id operation to e^e 
18. Write atory prob 
1 7. Create bar gra^, etc 
16. Draw conduelona from graph, etc 
13. Solve prob: est sum & diff 
11. Solve prob: place vafoe 
5. Relate mul & div facta 
3i Id at forms using regroupir^ 
1. Bd & compme patterns 




Wilson: 1993-98 CMT Math 4th Grade 
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Figure 25 identifies the specific mathematical skills that fourth grade students — ^who took the 
QVrr at Woodrow Wilson Elementary School between 1997 and 1998 — consistently scored 
low on and the percent of students vAio will most likely need assistance or intervention with 
these skills. 

Analysis of the charts on the 50 percent or more of fourth grade students who did not score 
at or above state goal between 1997 and 1998 on the CMT reveals that Wilson is 
e?q)eriencing a serious and consistent downward trend in students’ performance with the 
following specific mathematical skills: 



11. Solve problems: place value concepts 

16. Draw conclusions from graph, table, and chart 

18. Write story problems from number sentences 

19. Identify operation to solve story problem 

20. Solve story problem using addition and subtraction 

21. Solve story problem using extraneous information 
23. Identify needed information in problem situations 
28. Tell time 



Additionally, analysis of the charts on the 50 percent or more of fourth grade students who 
did not score at or above state goal between 1997 and 1998 on the CMT reveals that Wilson 
is e;q)eriencing a consistent low performance with the following mathematical skills: 



1. Extmid and complete patterns 

3. Identify alternate forms usang regrouping 

13. Solve problems: estimate sum and difference, including money 

17. Create bar graphs and pictographs from data 

18. Write story problems from number sentences 
21. Solve story problem using extraneous information 
23. Identify needed information in problem situations 

27. Estimate length and area 



Finally, de^ite a generally low-level performance between 1997 and 1998, analysis of data 
nevertheless indicates that Wilson is realizing substantial and consistent growth in the 
following skills: 



5. Relate multiplication and division facts to arrays and pictures 

18. Write story problems from number sentences 

21. Solve story problem using extraneous information 

24. Solve problems with elementary notions of pr<ril)abUity 
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H. Deficiencies Needing Attention Across all Schools 
The following list includes the specific mathematical skills that fourth grade students — ^who 
took the CMT at the district’s seven identified elementary schools between 1993 and 
1998 — consistently scored low on. 

CRE strongly recommends that the Connecticut Academy and the Waterbury Public Schools 
provide strategic technical assistance with the specific mathematical skills identified in the 
previous building-level analysis, for all K-5 regular classroom teachers. 

Additionally, CRE strongly recommends that the Connecticut Academy and the Waterbury 
Public Schools provide extensive, strategic, and consistent technical assistance to K-5 
teachers, wfio work in the seven elementary schools, in these following eleven mathematical 
skin areas for curriculum development, for performance-based assessment, and for 
professional development. Also, a substantial munber of the fourth grade students in these 
seven schools wiU most likely need assistance or intervention with development ofthese skills 
prior to taking the sixth grade version of the CMT in 2000. 



1. Extend and complete patterns 

3. Identify alternate forms using regrmiping 

5. Relate multiplication and division facts to arrays and pictures 

13. Solve problems: estimate sum and difference, including money 

17. Create bar graphs and pictographs from data 

18. Write story problems from number sentences 

21. Solve story problem using extraneous information (5 of 7 schools) 
23. Idoitify needed information in problem situations (6 of 7 schools) 

26. Estimate length and area (6 of 7 schools) 

27. Estimate length and area 

28. Tell time 13 of 7 schools) 
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ANALYSIS OF STUDENT PERFORMANCE DATA: READING 
Students’ reading ability is linked to mathematics most directly through the it ems on the 
mathematics test that are word problems. Thus, in an effort to support inq)rovement of 
students’ performance with mathematics, the Waterbiuy Public Schools requested that the 
Connecticut Academy focus some of its technical assistance on language arts. 

Figure 26 presents the analysis of foiuth grade students’ performance with the reading section 
of the CMT. Source data includes average percent of students meeting state goal for years 
1993, 1995, 1997, and 1998. Once again, CRE used the process of interpolation with the 
source data, in order to identify the average percent of students >^dio did not meet state goal 
within this time frame. 

Overall, Waterbury’s seven identified elementary schools performed similar to other schools 
in the same ERG. However, the comparison with other schools in the same ERG also shows 
that two ofWaterbuiy’s elementary schools had a greater percentage of students w^o did not 
meet state goal (Driggs and Wilson). Additionally, all seven schools have substantially greater 
percentages of students, \^o do not meet state goal, than the state average for this same 
period of time. Also, study of the performance records of these seven schools from year to 
year shows a pattern of gain/loss, gain/loss — concerning progress toward state goal The net 
result over time is a slight gain for Bucks Hill, HopevOle, and Walsh. For the other four 
schools, there is little or no gain. 



Figure 1 1 : Percent Below State Goal 




% Below State Goal 



CRE does not have documentation on students’ performance pertaining to specific reading 
skills. Nevertheless, on the basis of this chart, it is reasonable to conclude that all seven 
identified elementary schools in the Waterbiuy Public Schools would benefit from sustained, 
strategic assistance for the improvement of students’ reading ability. The schools’ increased 
growth toward meeting the state goal on reading skills will support students’ performance on 
the mathematics section of the CMT. 
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ANALYSIS OF STUDENT PERFORMANCE DATA: WRITING 
The linkages between performance on the writing and mathematics portions of the CMT are 
much less evident than between the reading and mathematics portions. Nonetheless, since the 
CSDE reports CMT data on students’ writing performance, CRE includes analysis of that 
data. 

Figure 27 concerns the percent of fourth grade students meeting state writing goal for each 
of the four years. Analysis of this chart shows that all identified elementary schools have 
realized steady growth in performance on the writing portion of the CMT during the 1993, 
1996, 1997, and 1998 school years. Indeed, Bucks Hill has exceeded the state average and 
the ERG average for 1998. 



Figure 12: Growth Toward State Writing Goai 




Q 1003 B 1000 Q 1007 I 1008 



On the basis of this analysis of data, CRE concludes that five of the seven identified 
elementary schools Waterbury district can take pride in their performance record relative to 
in^roving students’ writing skills consistent with the state goal — ^Barnard, Bucks Hill, Chase, 
Driggs, and Walsh. The two exceptions to this positive record are Hopeville and V^lson 
elementary schools. 

However, the inq>lications of students’ inq)rovement in writing for their performance on the 
mathematics portion of the CMT are difficult to determine on the basis of this data. In view 
of this analysis of CMT performance records in reading and writing, CRE strongly 
recommends that the Connecticut Academy and the Waterbury Public Schools enq>hasize 
more intensive anafysis of specific language arts skills, in order to support data based 
decision-making for the in^rovement of students’ reading in the content areas — especially 
mathematics — and ^dy skills, including a focus on students’ test taking skills . 
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ANALYSIS OF DATA: EDUCATORS 

A questionnaire was distributed to all of the teachers ^^o participated in SIST at the seven 
different elementary schools. The items on this evaluation instrument provided opportunity 
for controlled choice (ie., yes/no; multiple-choice) and follow-up written explanation of 
answers. SIST team members distributed copies of the evaluation instrument to the teachers. 
When teachers conq>leted the survey, they returned it by mail directly to CRE. The return rate 
for the survey was high: 71 percent. Most teachers provided additional written comments, 
>^Iiere requested on the survey. 

CRE also conducted structured intervie\vs with ^/yof the seven building principals and with 
the assistant superintendent. Wilson’s principal was on personal leave during the interview 
process, thus, CRE was not able to contact him. 

Nevertheless, docmnentation from the teachers’ surveys, in 
addition to material from partic^ating administrators and CRE’s observations, provide a 
sufficiently well-developed data base from ^\frich to draw conclusions regarding SIST’s 
effectiveness. Analysis of data indicates that there is agreement across the categories of 
particq)ants on the different survey items and interview questions. Exceptions are noted. 

The analysis of data pertaining to educators’ perceptions of the SIST initiative appears in the 
next four sections: Teachers’ Assessment of SIST’s Effectiveness, Administrators’ 
Assessment of SIST‘s Effectiveness, and SIST’s Self-Assessment. 
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i. Teachers* Assessment of SlST*s Effectiveness 

Return Rate: CRE documented an average return rate of 71 percent on the teachers’ survey. 
At Barnard, Bucks EBU, and Driggs the rate of return was at or above 100 percent. At Chase 
the return rate was above average. At Walsh and Wilson, the rate of return was sKghtly above 
average. At Hopeville, the rate of return was approximately 13 percent. 



Building Assignment; Distribution of survey respondents at the seven partic^ating schools. 



Boflifiiig 


#Classroom 

Teachers 


#sisrr 

Participants 


# Survey 
Respondents 


% 

Respondents 
per Building 


% Dlstiibiitioa 
per BuQcfin^ 


1 


Barnard 


13 


14 


14 


100% 


10% 


2 


BudcsHiU 


27 


28 


35 


125%* 


25% 


3 


Gbase 


33 


36 


30 


83% 


21% 


4 


Drxggis 


33 


29 


34 


117%* 


24% 


5 


Hopeville 


24 


24 


3 


13% 


2% 


6 


Waldi 


27 


29 


18 


62% 


13% 


7 


Wilson 


12 


11 


6 


55% 


4% 



Note: Asterisk (*) indicates that respondent indicates that the group of educators participating in SIST 
includes not only regular classroom teachers, but also specialists. 




Summary: Based on the percentage rate of buildings in the response group, teachers from 
Bucks IBll, Chase, and Driggs represent 75% of the response group. The remaining 25% 
conasts of responses from Barnard, Hopeville, Walsh, and Wilson. Ninety-nine percent, 
(99%), of the re^ondents are classroom teachers. The one exception is a first year 
administrator at Barnard School, who had been a classroom teacher for 23 years previous to 
the current assignment. 
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Gender: Eighty-six percent, (86%), of the survey re^ondents are female. Nearfy all the 
respondents, (89 -100%), fromDriggs, Hopeville, and Walsh Schools are female. The largest 
percentage of male re^ondents is 29% at Barnard School 



Fignre 14: Gender Distrilnitioa of Snivey Respondents 




Female 



Legend 
^ Male 



I I No Reiponae 



BEST CX)PY AVAILABLE 
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Grades Currendy Assigned: The following two charts concern the percentage of 
re^ondents assigned to teach at each grade level from Pre-K to fifth grade. In most sdiools 
the grade level distribution is roughly equal, with the exception ofWalsh, \^dlich has a greater 
percentage ofprimary grade teachers re^onding and Wilson, at which the only representation 
in the primary grades is from the Pre-Kteachers. Total percentages exceed 1 00, because some 
re^ondents are re^onsible for teaching more than one grade. 



Figure 15: Grade Level Asagnment of Respondents 
Orado Ldvala 

M M ^ ^ B4 




Grade Level Assigpnieot cf Re^oudoxts 


Reqionse Groups 


Uot 

Respondents 


Grade Levels 


Pre-K 


K 


1 


2 


3 


4 


5 


All Respondents 


140 


5% 


2m 


20% 


21% 


20% 


20% 


17% 


Barnard 


14 


0 


22% 


22% 


15% 


15% 


15% 


15% 


BudcsHill 


35 


6Vo 


17% 


20% 


14% 


11% 


20% 


17% 


Chase 


30 


0 


23% 


23% 


27% 


30% 


27% 


23% 


Driggs 


34 


12% 


30% 


18% 


21% 


18% 


18% 


9% 


Hopeville* 


3 


0 


100% 


em 


67% 


33% 


33% 


33% 


Waldi 


18 


0 


33% 


17% 


28% 


22% 


6% 


6% 


WilSGD 


6 


17% 


0 


0 


0 


33% 


33% 


33% 



Note: Asterisk reminds readers that percent distribution is based on only a total of three respondents from 
Hopeville. 
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ffighest Degree: The followmg chart addresses the percentage of re^onding teach^ who 
have reached particular degree levels. The largest percentage of re^ondents in each school 
has completed a Masters program and beyond. Chase has the larged munber of teachers >^o 
have recently received the Bachelors degree. 



BA/BS 

DoclMBte 



Figme 16: H^iest Degree Received by Respondents 

■ MA/M8 n MiVMS+lS 

Bi PnBaqgadKBioed degree 




Note: Zero (0) indicates no response recorded by respondents in that category. 
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Years of Teaching: The average niunber of years of teaching e?q>erience per building varies 
from 9 to 23 years, with Barnard School having the most e 7 q>erienced teachers and Wilson 
having teachers with the least numher of years of experience. 



Figure 17: Avenge Nnmbei of Years ofTeadungE^xtieDce 
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How many SIST Workshops for language arts and math did you attend? Teachers who 
re^onded at every school attended at least one workshop in each conqronent. Teachers at 
Barnard attended the greatest munber ofworkshops. Documentation indicates that a majority 
of Barnard’s respondents attended five or more workshops in each con^onent. 



Fignte 18: Attendance at Language Arts Wotkshops 
Number of Workshops 

^ 1-2 9 3—4 5+ ^ Ko Roiposiu 




AERMposidents Barnard BuoksHiU Chase Dnggs HopeviUe Walsh Wilson 



FigDie 19: Attendance at Math Workshops 
Number of Workshope 

^1-2 ^3-4 n ^ @ KoRmpoou 



100 -I 




AE Respondents Barnard Bnokalfill Chase Dxiggg HopeviUe Walsh Wilson 



Note: zero (0) indicates no response recorded!^ respondents in that category. The category of “No Response” 
for the above charts indicates that either teachers did not enter a response or that the question was not 
applicable to their situatioa Also, totals in both charts do not always equal 100%, due to the rounding of 
numbers. 
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How many times did you take the opportunity to watch the language arts and math 
consultants teach? Although less than half of the survey re^ondents answered this item, 
those that did provide answers indicated that they had watched the consultants teach at least 
once during the initiative. More than half of the responding teachers at Bucks IBll, Giase, and 
Wilson indicated that they had watched the language arts consuhant teach. A similarly high 
percentage of responding teachers at Barnard, Bucks IBll, and Chase took the opportunity 
to watch the math consuhant teach. 



Figure 20: Opportunities Taken to Watch Language Arts Consultant Teach 
Number of Tlmeo Obton/ed 
H 1-2 H 3-4 □ 5+ 




AH Reyrndimte Banurd Bodu IfiU Chace Dtiggi Hopeville Waldi Wikon 



Figure 21: Opportunities Taken to Watch Math Consultant Teach 
Number of Tlmeo Oboerved 

^ 1-2 ® 3-4 □ 54 - 




AH Reqxmdentft Barnard Books Hill Chase Driggs H^eviHe Waldi Wilson 



Note: Zero (0) indicates no response recorded respondents in that category. Also, the response from all 
respondents was less than 50%, but analysis of data within individual buildings shows that in some schools 
the response rate on this item is high. In contrast, within Hopeville, the response rate is zero. 
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How often did yon meet with the consultants to discuss the worksh<^*s implications for 
your classroom teaching? At five of the elementary schools, less than 40% of the 
respondents answered this hem. Only at Barnard and Bucks Hill did more than 50% of the 
respondents provide data. The average amount of time that teachers at Barnard and Bucks 
HQll said that they discussed the inq)]ications of the workshops for classroom teaching was 3 
times. 

OveraD Assessment of the SIST Workshops: In general, the majority of aU respondents 
were in agreement with each hem concerning overall assessment. Analysis of data within the 
individual buildings ^ows that most respondents at Barnard and Bucks Hill agree with each 
item concerning overall assessment. Conversely, few ofthe respondents at Driggs agreed with 
this overall assessment of SIST workshops. 



Percentage of Res|>ondents in Agreement with Each Item 


Survey Group 


The SIST Workshops were: 


vpAo- 

date 


timely 


well- 

organued 


wdl-managed 


efifectiveLy 

in^lemeated 


meeting my 
needs 


All Respondents 


73% 


62% 


55% 


60% 


55% 


54% 


Barnard 


93% 


93% 


86% 


93% 


93% 


86% 


Bucks Hill 


97% 


98% 


92% 


94% 


95% 


94% 


Chase 


73% 


46% 


27% 


43% 


70% 


27% 


Driggs 


32% 


18% 


21% 


24% 


18% 


15% 


Hopeville 


67% 


67% 


67% 


67% 


67% 


33% 


Walsh 


94% 


89% 


72% 


72% 


66% 


77% 


Wilson 


50% 


34% 


50% 


17% 


17% 


50% 



Teachers’ comments on overall assessment of SIST workshops: In written comments, 
teachers identified linda Ball and Pamela Barker Jones as effective math consultants. A list 
of common terms used by teachers to describe the work of Ball and Jones includes: very 
productive, informative, interesting, down-to-earth, an excellent resomce, gave great ideas, 
was always available or accessible, and very helpful. Additionally, teachers repeated these 
positive terms when they were asked to identify what were the strengths of the math 
con:q)onent. 

In their written comments, teachers reported that the weaknesses of SIST’s math conq>onent 
are: organization of the program and scheduling of activities with schools was inconq)lete, 
awkward, and ineffective; too little sustained time in the buildings; substitute teachers were 
not always available; presentations were not effectively organized; consultants don’t &ce 
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difficulties of disciplining children and, thus, sometimes come across as out-of-touch with the 
realities of urban education. 

In contrast, many teachers identified Joe Cellizza as less-than-efifective as the language arts 
consultant. A list of terms fi'equently used by teachers to describe Celliza’s work includes: 
worthless workshops, out of touch with CMT needs, not enough substance, too much time 
focused on retelling stories, dry, and too philosophicaL 

Paradoxically, some teachers e?q)ressed modest satis&ction with both the math and language 
arts components, some teachers were pleased with worktops presented by Joe Cellizza, and 
a substantial group of teachers e?q)ressed very strong dissatis&ction with both the math and 
language arts conq)onents. Satisfied teachers eiq)hasized in general terms the usefiilness of 
the methods and materials for their classroom practice. 

Dissatisfied teachers wrote that they were not &vorably inq)ressed with any ofthe workshop s; 
saw no evidence of a strategic plan fi-om SIST for building-level technical assistance; saw no 
obvious links with previous building-level professional development activity; reported that 
some of their colleagues would be better presenters; reported that worktop presenters 
should have immediate, practical e?q)erience with teaching the grades for which their 
worktops apply. Comments fi-om dissatisfied teachers came fi-om all ofthe diSerent schools, 
but were especially co mm on to buildings w^ere the analysis of quantitative data also reveals 
a strong, critical perspective regarding SIST. 

Finally, teachers stressed that the Connecticut Academy should have been more careful in its 
hiring of consultants — especially concerning the assessment ^ecialists (the Johnsons) — ^in 
order to present assessment effectively to the teachers. Additionally, teachers noted that, 
when the Johnsons were removed fi-om the program, the Connecticut Acaden^ should have 
continued the assessment conq)onent with other, qualified personnel, instead of dropping it 
fi-om the program. Also, teachers noted that the absence of an orienting and sustained focus 
on student performance data in reference to teachers’ professional development meant that, 
in the end, the teachers cannot tell how to determine or what should be done to in^rove 
students’ performance on the CMT. 
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How would you assess the overall effectiveness of these workshops? Teachers were asked 
to reflect on SIST’s overall efiectiveness. Across all the survey respondents, 17% assessed 
the workshops’ overall efiectiveness as excellent, 27% assessed them as good, 19% assessed 
them as average, and 21% assessed them as poor. The data fi-om individual buildings shows 
that a substantial percentage of respondents assessed the workshops as good to poor. 



Figure 22: Overall EfiBsctiveness of Wodrdiops 

^ Rtorflcpl B <ipod O Avc(%d ^ Poor 




Note: Zero (0) indicates no response recorded by respondents in that category. 
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SIST Consultant's Efiectiveness: Analysis indicates that a majority of all respondents 
agreed that consultants were enthusiastic about their topics and eager to he^. The area of 
consuhants’ greatest weakness was in their demonstration ofinstructional technology. Within 
individual buildings, teachers at Barnard and Bucks HQl e?q)ressed a more positive view of 
the consuhants’ effectiveness than teachers at the other five elementary schools. Teachers at 
Diiggs expressed the least positive views of consuhants’ effectiveness. 



Percentage of Respondents in Agreement with Each Item* 


Survey 

Group 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


All Respondents 


67% 


73% 


75% 


66% 


64% 


61% 


64% 


67% 


43% 


61% 


Barnard 


93% 


100% 


100% 


86% 


93% 


93% 


100% 


100% 


79% 


92% 


BudksHill 


94% 


97% 


95% 


91% 


94% 


97% 


94% 


89% 


86% 


94% 


H.S. Chase 


47% 


67% 


73% 


56% 


53% 


80% 


66% 


70% 


60% 


50% 


Driggs 


33% 


33% 


36% 


33% 


27% 


27% 


24% 


24% 


9% 


33% 


Hopeville 


100% 


100% 


66% 


100% 


33% 


33% 


33% 


33% 


0 


0 


Waldi 


83% 


83% 


83% 


78% 


'77% 


72% 


55% 


80% 


55% 


61% 


Wilson 


67% 


100% 


100% 


50% 


50% 


34% 


67% 


84% 


34% 


89% 



*The SIST Consaltants were: 



1 Effective with present^ticii skilM 

2 Enthusiastic about the topics? 

3 Eager to he^ you? 

4 Well prepared, in regard to relevant oonteatknoMdedge? 

5 Equipped with good teaching ideas -for your purposes? 



6 Effective with classroom modeling? 

7 Effective vAim interacting with different people in sdiool? 

8 Effective when demonstrating use of manipulative? 

9 Effective vAm demonstrating use of tedmtdogy? 

10 Effective when adfptingto schedules and school issues? 



BEST COPY AVAILABLE 



Onricuhim Research & Evaluation, 237 Singleton Road, Qi^lin, CT 06235#860>455-1229#Fax 860*455-001 14www.creus.ocHn 



1999 SIST Evalution: Waterbury Page 36 



Impact on Teaching: Analysis of data across all le^ondents shows that more than 50% of 
the teachers perceived the methods and materials that were presented at the workshops to 
be new or that they were fresh and understandable. Analysis of data within individual 
buildings shows that a majority of respondents from Barnard, Bucks Hill, and Walsh 
e?q)ressed similar positive perceptions regarding the methods and materials introduced in the 
workshops. However, at Giase, Driggs, HopevOle, and Wilson less than half of the teachers’ 
reported that the methods and materials were new or that they were presented in a fresh and 
understandable format. 



Percentage of Re^ndents in Agreement with Each Item 


Survey Group 


The SIST workshops: 


were filled with new methods 
and materials 


rvitainftH famnirtr mcdiods and 
materials presented in a fresh 
and understandable format 


All Respondents 


55% 


53% 


Barnard 


86% 


93% 


Bucks Hill 


80% 


83% 


Chase 


44% 


26% 


Driggs 


27% 


27% 


Hdpeville 


33% 


33% 


Walsh 


72% 


66% 


Wilson 


17% 


34% 
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Have you started using some of the methods and materials you have encountered in the 
workshops? Across all re^ondents, analysis of data shows that a majority (63%) of the 
teachers have begun using the methods and materials presented in the worktops. Analysis 
of data within schools shows that at Dtiggs and Hopeville less than half of the teachers are 
currently making use of this information. 



Figote 23: C uiroit Use of Ptesenled Methods and Matenals 
Am you eurrentty using tftls intannallen? 




All Re^ondants Barnard Bocks Hill Chase Dtiggs HopefviHe Waldi Wilson 



Note: percentages for the above items do not equal 100%becausenoteveiy respondent answered the question. 
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How much of the SIST methods and materials do you plan to use next year? Analysis 
of data shows that approximately 50% of all re^ondents antic^ate that, next year, they will 
use from 25 to 50% of the SIST methods and materials. Thirty-one percent (31%) of all 
req>ondents reported that they antic^ate using between 75 to 100% of the methods and 
materials. Analysis of data within buildings shows that at Barnard and Bucks HUl half of the 
req>ondents antic^ate using between 75 to 100% of these methods and materials. However, 
at the other five elementary schools a majority reported that, next year, they anticipate that 
they will use only 25 to 50% of these methods and materials. Additionally, at Hopeville and 
Wilson, 33% of re^ondents indicate that, next year, they willnot be using any ofthe methods 
and materials presented at the workshops. 



Figure 24: Future Use of Presented Methods and Materials 
^ looy# @ 75% □ 50% H 25% ^ 



50 

40 




AHRMpondotf Baxnaard Bnokilfill Chasb Dnggi Hopevine Wahh 

Note: Zero (0) indicates no response recorded by respondents in that category. Also, percentages for the above 
items do not equal 100% because not every respondent answered the question. 
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Assess the workshops contribution to your professional devdopment: The analysis of 
data from all respondents shows that a majority of all re^ondents found the workshops’ 
contribution to their professional development to be from good to average. At Barnard 21% 
and at Bucks Hill 23% of the respondents found the workshops’ contribution to be excellent. 
Conversely, in the remaining buildings from 22% to 50% of the respondents indicated that 
the initiative’s contribution to their professional development was poor. 



Figure 25: Woikshc^s Contiibutioiis to Profesdonal Develf^unent 

^ BkoeUent ^ Oood Q Ara%e @ Poor 




AH Rogpondfflite Barnard Bncki Hill Chaeo Dxiggg Hopeville Walsh Wilson. 



Note: Zero (0) indicates no response recorded by respondents in that categoiy. Also, percentages for the above 
items do not equal 100% because not eveiy respondent answered the question. 
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Do you have any evidence, thus far, that these methods and materials are improving 
students achievement? The chart clearly indicates that more than half of the respondents in 
all of the buildings do not have any evidence that the workshops’ methods and materials are 
in^roving students’ achievement. 



Figure 26: Ixopact on Student Achievement 
Do you have ony ovidance of Impact? 

P Yoi @ No 




All RoQ>ondantg Barnard Bnclcs IfiU Chase Dnggs Hopevillo Walsh WiUoii 



Note: Zero (0) indicates no response recorded by respondents in that category. Also, percentages for the above 
items do not equal 100% because not every respondent answered the questioa 
Teachers’ comments on SiST’s impact on students learning: In general, teachers 
indicated that they do not e?q)ect that the SIST workshops will have much, if any, intact on 
students’ learning. The most immediate positive inq)act on students’ learning that teachers 
reported was that students appeared to he more motivated and more interested in doing math, 
for example, with the man^ulatives and related hands-on activities. Clearly, some teachers 
had been r^venated by the workshops and said that they hoped that their own excitement 
about the methods and materials would have a positive inq)act on the children’s learning. 

However, many teachers reported that they had been introduced to these methods and 
materials previously, thus, they saw very little if anything “hew” in the workshops. In other 
words, more of the same classroom instruction would not contribute to significant changes 
in students’ learning. Others reported that they had in^lemented these practices prior to 
SIST. The teachers who expressed critical co mme nts described the consultants’ workshops 
as redundant and the presenters as not particularly well-informed about the staffs strengths 
and weaknesses or the students’ CMT performance records. Thus, they reiterated that SIST’s 
intact on students’ learning was doubtful. 

Additionally, many teachers — ^including those who were generally pleased with the 
worktops — repeated their fiiistration and disappointment with SiST’s organization, 
scheduling, use of substitute teachers, and loose coupling with building needs. Many teachers 
recognized and applauded the idea of strategic professional development. They were 
disappointed with its implementation in their buildings. 
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Contribution to Building-Levd Improvement: Analysis of data across all re^ondents, 
shows that the only item on wiiich more than 50% of the teachers e>qpressed agreement 
concerned the initiative’s increase of budding-level eTqpectations regarding professional 
development. Additionally, data shows that SIST did not have a substantial intact on 
building-level in^rovement, except at Barnard and Bucks HSL Data indicates that Barnard 
and Bucks HSl are the only schools that have a substantial percentage of teachers for whom 
SIST created a strong expectation that the initiative should continue in the future. 



Percentage of Respmdents in Agreement with Each Item 


Survey 

Group 


The SIST workshops; 


Bicreased building level 
e?qiectations regarding 
prctessioDal developmoit. 


Created a favorable 
reeponse among all 
teachers. 


Created a favorable 
response among all 
teadios 


Created a strong 
oq)ectalion to ocDthme 
SICT in the future 


All Refipcndcnts 


52% 


45% 


47% 


39% 


Barnard 


93% 


87% 


93% 


86% 


Bucks Hill 


85% 


82% 


80% 


77% 


H.S. C3iase 


43% 


37% 


40% 


17% 


Driggs 


17% 


12% 


18% 


9% 


HopeviDe 


33% 


0 


0 


0 


Waldi 


39% 


28% 


17% 


28% 


Wilsdi 


50% 


34% 


51% 


34% 
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Assess the overall buildmg-4evd response to SIST wwkshops: Analysis of data across all 
re^ondents shows that approximately half of the teachers (49%) found the SIST workshop s 
to be average to poor. This perception was also true of data analysis within buddings, with 
the exception of teachers at Barnard and Bucks HdL 



Figure 27: Overall Bmlding-Level Response to Woiluliops 

^ BKodlent B Qoo6 O Avenge ^ 



70 n 



67 



60 - 




AU RMpoodoiIs Bamanl Books IfiU Driggi HopoviUd Woldi Wilsoii 



Note; Zero (0) indicates no response recorded by respondents in that category. Also, percentages for the above 
items do not equal 100% because not every respondent answered the question. 
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Should SIST workshops continue in the immediate future? Analysis of data across all 
re^ondents shows that 46% felt that this initiative should continue in the immediate fiiture. 
However, analysis of data within b uilding s ^ows some interesting contrasts. At Barnard and 
Bucks Hill there is a great majority of respondents who said that the workshops should 
continue. At Walsh and Wilson, on the other hand, re^ondents are equally divided on the 
question of SIST’ s continuation. Additionally, at Chase, Driggs, and Hopeville, the majority 
of respondents said that SIST worktops should not continue. 



Figure 28; Sihonld the SIST Workshops Contiiuie? 

^ Ym ^ No 




AH Ra^andcnti Barnard BnekiiHl Chaae Dngga Hopeville 




Waldi 



SO SO 




Wilaoo 



Teachers’ comments on continuation of SIST workshops in the future: Approximately 
25% of the teachers said that continuation of SIST workshops in the future would be 
beneficial for their classroom practice. They said that SIST would keep teachers up-to-date 
on usefiil methods and materials. 

A large majority of teachers expressed dissatis&ction with SIST’s operations, personnel, 
content, and scheduling. A small number of teachers bluntly described SIST as a waste of 
time and money for the Waterbury Public Schools. 

However, teachers who e?q>ressed constructive criticism were more common in the data set. 
These individuals noted that they would be interested in SIST’s continuation, but it would 
have to change its program substantially, in order to appropriately address their instructional 
needs. These critics oflFered specific and relevant recommendations for irrq)roving SIST. 
fiideed, as the following list from the teachers reveals, there is substantial overlap with 
recommendations from the school administrators. (See next section for comparison.) 
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Teachers* Recommendations for Continuation of SIST: Included among the teachers’ 
recommendations for SIST’s continuation are; 

• begin workshops earlier in the school year 

• mq)rove communications and avoid conflicts with teachers, princ^als, and central 
office 

• eliminate scheduling problems by adopting a design that works for each building 

• design worktiiop s for ^ecific grade levels; do not hm^ all teachers together, because 
pre-kindergarten, kindergarten, primary grade teachers, and intermediate level 
teachers have very different instructional programs and needs for technical assistance 

• prior to in^lementing the program, clarify the program’s purpose, prepare a ^ecific 
plan targeted to each building, and publicize this plan for teachers and princ^als 

• work with a smalle r number of buildings and enq)hasize accountability and 
effectiveness for SIST prior to moving to other elementary schools 

• provide more days and more sustained time periods for quality inservice 

• bring to teachers many genuinely new ideas that are practical and relevant to their 
worlq)lace circumstances 

• design and inq)lement grade-level-specific intervention strategies for teachers to use 
to help inq)rove students’ performance on the CMT 

• organize more grade level meetings for open discussion of instruction, wliat works, 
what does not work, and problem solving 

• organize the program for consistent support from substitute teachers. 
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2. Administrators’ Assessment of SIST’s Effectiveness 

Documentation in this section is from interviews that CRE conducted with building principals 
assigned to Barnard, Bucks Hill, Qiase, Driggs, HopeviDe, and Walsh elementary schools, 
and with the assistant superintendent. The interview questions pertain to the contract that 
Waterbury Public Schools signed with the Coimecticut Academy. 

Briefly, analysis of data indicates that there is a high level of agreement across the different 
school administrators. These administrators stressed that they are in regular contact with each 
other and that, as a result, their perceptions of the issues associated with SIST are commonly 
understood and well-supported. Additionally, administrators e7q>ressed strong support for the 
SIST concept. They explained that a strategic, consistent, and sustained building-level 
approach to professional development is the key to their success with uxq)roving students’ 
performance with basic skills, especially those that are emphasized by the CMT. The assistant 
superintendent said that he chose these seven elementary schools for SIST because these 
schools need assistance and their staffr are in a good position to employ data based decision- 
making for curriculum development, assessment, and professional development, in order to 
ni 5 )rove students’ performance on the CMT. 

Thus, Waterbury school administrators welcomed assistance from SIST. Additionally, they 
provided an opportimity for the intervention program to imfold in their schools. However, 
analysis of data shows that administrators assessed SIST’s overall performance as mixed 
(some high marks and many low marks), with an overall rating of six on a ten-point 
scale — slightly above average. 

hi the following section, there is summary documentation from each of the questions used in 
the interviews with administrators. 

A. Status of the Elementary School Curriculum 
To i\hat extent are the elementary mathematics and language curricula comprehensive, that 
is. developed fully for all grades? Administrators report that the Waterbury Public Schools 
convicted the K-5 mathematics curriculum during the current school year and had conqileted 
the K-5 language arts curriculum dining the 1 996-97 school year. Each school has the formal 
curriculum posted clearly and prominently, for practitioners and visitors to read, hi fell 1999, 
each classroom will have its own poster of the fiirmal curriculum 

Are the elementary mathematics and language arts curricula based on the Connecticut 
curricuhim frameworks? Yes. Administrators report that the mathematics and language arts 
curricula are based on national and state standards and on the Coimecticut curriculum 
frameworks. 

B. Status of Elementary Teachers* Practice Relative to Curriculum 
To i\hat extent are elementary teachers’ instructional practices in mathematics and language 
arts are consistent vrith the curricula in these areas? Administrators reported that the extent 
to which elementary teachers’ instructional practices in mathematics and language arts are 
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conastent with the formally adopted curricula in these areas varies by the building and by the 
individual teacher. However, princ^als stressed that, in general, their teachers follow the 
newly adopted curricula. Princ^als also reported that teachers are more likely to follow the 
curricula now than in the recent past, because of the specific detail in the curricula, the 
obvious linkages with objectives in the CMT, and the current opportunity for building-level 
analysis of CMT assessment data. Additionally, principals report that the teachers are highly 
interested in assistance with materials and methods that will help them to hiq)lement the new 
curricula. 

Principals and the assistant superintendent said that they had expected SIST to help teachers 
iir^rove their ability to match instructional practices with students’ needs — consistent with 
individual building needs — and to provide up-to-date, challenging, and motivating materials 
and methods to the students. However, administrators reported that, for a variety of reasons, 
they were disappointed with the in^>lementation of SIST. Chief among their concerns were 
shortcomings which they observed in the following areas: purpose, organization, scheduling, 
appropriateness of content, and style. (For more discussion of this issue, see sections E and 
F below.) 



C. Faculty Transfer/Retirement Rate 

What percentage of elementary faculty transferred out of schools during 1996-97. 1997-98. 
and 1998-99? Administrators reported that feculty transfer and retirement occur at very low 
levels in these seven elementary schools. Thus, staff turnover at the building level is low to 
nonexistent, contributing to a more or less stable environment, as &r as stafBng is concerned. 

D. Assessment of Students* Performance 

How do the administrators assess students’ current performance in mathematics, as measured 
by the CMT? Administrators reported that students’ performance on the mathematics portion 
of the CMT is low across all elementary schools, but that it is consistent with the performance 
of students who attend public schools in Connecticut’s other urban communities. 

These administrators also said that the consistently low performance rating, especially on 
some objectives or specific skills, is ample justification for employing a strategic approach to 
teachers’ professional development. The administrators said that they were in fiill agreement 
with the idea behind SIST. However, they reiterated that they were very concerned about its 
inq)lementation, \^fiich they said fell short of their expectations. 

How do the administrators assess students’ current performance in language arts, as measured 
bvthe CMT? A dmini strators reported that students’ performance on the language artsportion 
of the CMT was low, which is consistent with performance ratings in other urban 
communities in Coimecticut. However, they reported that students’ performance in some 
instances (for example, Barnard) exceeded the city’s average. This positive result the 
administrators attributed to their concentrated attention to in^)roving the language arts 
curricuhim, consistent with the CMT and cimiculum frameworks. Although administrators 
e?q)ressed disappointment with both math and language arts components of SIST (math at 



Curriculum Research & Evaluation, 237 Singleton Road, Chaplin, CT 06235#860-455-1229#Fax 860-455-001 1 ♦www.creus.com 



1999 SIST Evahition: Waterbury Page 47 

Driggs and Hopeville was the exception), they were especially disappointed with language 
arts. 

E. Assessment of SIST*s Effectiveness 

What is the overall assessment of SIST? In other words, how effectiv e was this professional 
development program at meeting the elementary teachers’ instmctionalneeds? Administrators 
assessed SIST’s overall performance as mixed. A summary of the administrators’ assessment 
follows. 

• Math workshops were much better than the language arts workshops. 

• Asa result of the mixed performance from the technical assistance providers, teachers 
and administrators recognized the inqiortance of focusing their attention on analysis 
ofCMT data and taking responsibility themselves for planning a strategic professional 
development program for their buildings. 

• There was good coverage for the fourth and fifih grade teachers, but little or nothing 
for the Pre-K to third grade teachers. 

• Presenters did not adequately study building-level CMT records, in order to model 
data based decision-making and to provide workshops based on building-level needs 
(Driggs and Hopeville were exceptions). 

• Some presenters cooperated with principals and teachers, others did not, v^ch led 
to misunderstanding about the purpose, scheduling problems, and frustration. 

• Frequently, workshops included outdated materials and familiar methods. 

• In some instances, the workshop presenters’ language and mannerisms were perceived 
by teachers and administrators as demeaning, insulting, and counterproductive. 

• The assessment conq>onent — ^perceived by administrators to be the most inqjortant 
element — ^failed from the beginning because the presenters’ (the Johnsons) used a top- 
down, authoritative approach. This conq)onent was discontinued. The Johnsons were 
described by administrators as poorly informed about urban education, “acting as 
though they were better than anybody else in education,” and e?q)ecting Waterbury’s 
teachers and administrators to “toe the line.” 

Did SIST accomplish its purpose at a satisfactory level? A dmini strators gave SIST a qualified 
score of 9 X, in a ten-point scale, indicating that it was slightly above average with respect to 
^^t they classified as “satisfectory.” The administrators said that they were expecting an 
outstanding performance from SIST. 

To wbiat extent did the mathematics and language arts components of SIST accomplish their 
purposes? Administrators described the mathematics conq)onent as excellent at Driggs and 
Hopeville, especially for the upper grades. The workshop presenter knew her material, 
adjusted the workshops to the teachers’ varying needs, used excellent communication skills, 
and consulted regularly with the principals. The administrators assessed the math conqjonent 
at the other schools as satisfrctory for fourth and fifih grade teachers and not effective for 
Kindergarten, first, and second grade teachers. 

The language arts conq>onent was perceived by administrators and teachers as disappointing 
and a waste of time for everyone. 
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How did the teachers respond to the mathematics and language arts components of SIST? 
Administrators said that all of the teachers were somewiiat apprehensive about the math and 
language arts con:q)onents, a rehouse which they would naturally have toward any new 
program or outside agency. The administrators said that, despite the apprehension, their 
teachers were looking forward to receiving the technical assistance that was promised. 

At Driggs and Hopeville the teachers immediately accepted the math workshop presenter and 
worked with her as often as their schedules allowed. In the other schools, the mathematics 
workshops were basically accepted by the teachers. 

However, administrators said that \^dien the teachers discovered vsliat the language arts 
presenter offered, they went back to their classrooms and would attend workshops only vsdien 
required to do so. The administrators explained that the teachers re^onded in this manner 
because they already knew this material, they did not need theory enq)hasized to such an 
extmt, most of the materials were outdated, and they could do as well or better themselves. 
Administrators said that they concurred with the teachers’ opinion of the language arts 
consultant. 

What are the most valuable aspects of SIST? Administrators said that the most inq)ortant 
lesson from SIST is that they and their teachers could understand better that their schools 
need a sustained, consistent, and strategic approach to professional development. In other 
words, subsequent to the in^lementation of SIST, the teachers and principals learned the 
inq)ortance of focused, strategic professional development for tackling the issues they faced 
in cimicubun, instruction, and assessment. 

Additionally, principals said that, as a resuk of watching SIST unfold this past year, they and 
their sta£& recognized their own strengths and resolved to utilize teachers, who worked in 
their buildings, for workshops and co-teaching. Also, principals said that, when the SIST 
consultants were effective with organizing discussion sessions with teachers, it gave the 
teachers a good opportunity to meet and discuss common problems with regard to improving 
students’ performance on the CMT. Princ^als said that they recognized the inq)ortance of 
these peer group discussions and that, in the future, they will provide more opportunities for 
“sh and talk” sessions, where teachers will discuss co mm on issues and adopt effective 
solutions. 

Administrators commended Pamela Barker Jones for her exceptional ability to motivate 
teachers and to present use&l methods and materials for mathematics instruction. They said 
that Ms. Barker Jones provided the exanq)le of technical assistance that everyone was 
e7q>ecting in the SIST program 

What, if any, are the weaknesses or issues associated with SIST? In general, administrators 
were not pleased with SIST’s implementation. The major concerns about SIST that 
administrators identified were: 

• poor preparation for implementing SIST at different buildings — e.g., program did not 
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enq)hasize or model data based decision-makmg for technical assistance, and created 
instead the impression of SIST as a casual approach to professional development. 

• unclear statement of purpose for SIST 

• lacking direction for SIST from the beginning of its implementation to the conclusion 

• too few organizational meetings between principals and representatives from the 
Connecticut Academy for the purpose of formulating a specific, working plan 

• program hired consultants wiio lacked e?q)erience with urban schools, especially Pre- 
K-5 

• program was not revised nor were effective personnel employed wiien principals 
voiced conplaints 

• some consultants provided a weak response to individual teachers and principals, and 
the prevailing circumstances at the various sites 

• consultants didnot coordinate and update workshop schedules routinely with building 
principals 

• some consultants used negative speech toward teachers and administrators 

• some consuhants were inappropriately matched to grades (especially grades K-3) 

• some consultants used outdated materials and methods (especially in language arts) 

• consultants who were responsible for the assessment conponent got off to a had start 
and the program did not continue this aspect of SIST. 

Additionally, a majority of the administrators described Joe CeUizza’s performance with 
technical assistance in language arts as too academic. Analysis of data from teachers supports 
the administrators’ assessment of the language arts conponent. 

Will you rely on SIST for vour teachers’ professional development next year? The 
admioistrators stated enphaticaUy that they do not want SIST to return to their buildings with 
the same personnel, the same content, and the same processes that were used during the 
1998-99 school year. In general, this approach did not work well in the elementary schools. 
The a dminis trators stated that they recognize fuDy the integrity of the SIST concept. They 
said that this is exactly the kind of professional development that they need. However, they 
would rather do this work themselves, than have the SIST program repeated. Administrators 
said that they will welcome SIST’s return, if the following conditions are met by the 
Connecticut Academy: 

• consultants are highly qualified for the job 

• the revised SIST program is consistent with building- and district-level expectations 

• planning for the new SIST program responds effectively to the district’s critique. 

F. Recommendations for Improvement 

What recommendations can you offer to improve the SIST promam for the Waterbury Public 
Schools? Administrators offered the following specific recommendations for improving SIST. 
There is no order of inportance inplied in this list. 

Administrators said that SIST should do the following: 

• Emphasize strategic planning in advance of inplementing the program in a particular 
building. 
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• Accentuate ongoing collaborative leadership with principals, for identification of 
building-level needs and follow-through. 

• Formally design the SIST program for building-level technical assistance, make it 
clear to teachers and administrators wdiat the program entails, and inq)lement the 
program according to the plan. 

• En^hasize \^at works. Demonstrate methods and materials that he^ urban children 
to learn reading, writing, and/or mathematics. 

• Hire top quality consuhants with expertise in specific areas of need. 

• Enq)loy better organizational skills for more effective programming, hiring, personnel 
management, scheduling, and access to or encoimter with the local she. 

• T ake into account the different sizes ofbuildings (i e. , student enrollment and &cuhy 
levels) vriien planning the program. Be more flexible and inventive — don’t adopt the 
same general plan for every building regardless of hs size. Make the SIST program 
equhable for all buildings. 

• Maintain better control of substitute teachers, including using alternative strategies 
wiien substitute teachers are not available or for A\dien they are in short supply. 

• Provide better exanq)les for using CMT data strategically in order to plan and to 
inq>lement effective lessons and to conduct follow-up activity, including assessment. 

• Organize the building-level SIST program for maximiun positive inq)act on all of the 
teachers, within a sustained period of time. Do not make workshops and other events 
optional without the principal’s agreement. Do not plan on working with teachers fi)r 
one or two days per week, as was done in 1999. This is an insufficient amoxmt of 
time, is counterproductive in the long term, and sacrifices both continuity and 
momentum. 

• Organize and facilitate a sufficient number of discussion sessions, during wfiich 
teachers describe common issues and propose effective solutions. 

• T eachers and consultants need to develop respect for one another, vriiich takes finely 
developed personal relations skills on the part of the consultants. Also, h takes time 
to develop trust. 

• Do not enq)loy the hit-and-miss approach, suggesting that topics are up to chance or 
that any topic, lesson, or material will do for all children in a class or for all grades. 

• Consultants and teachers should collaborate for teaching; for sharing and modeling 
sessions; and for resolving issues, such as scheduling or substitute teachers. 
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3. SIST*s Self-Assessment 

In this section, there is documentation pertaining to the consultants’ performance with the 
SIST initiative in the seven identified elementary schools. Main sources of data are self- 
reports prepared by the consultants; interviews and discussion sessions with representatives 
from the Connecticut Academy; and observations of business meetings, during which 
consultants, representatives fi-om the Connecticut Academy, and personnel from the 
Waterbury Public Schools discussed SIST’s operations in the Waterbury schools. 



In preparation for consuhants’ completion of self-reports, the princpal evaluator developed 
evaluation questions for consultants to answer and, also, requested that they provide as much 
detail as possible, including supporting material (ie., handouts and schedules). 



The general plan for SIST gave each consultant responsibility for providing ten days of on- 
site technical assistance to one or more of the seven elementary schools. The following list 
presents the budding assignments of the SIST consultants, during spring 1999. 



CONSULTANT 
Linda Ball 

Pamela Barker Jones 
Mj Terry 
Joseph Cellizza 



SPECIALTY BUILDING 

Mathematics Bucks Hill 

Mathematics Driggs, Hopeville, Walsh 

Mathematics Chase, Barnard, Wilson 

Language Arts All seven schools 



A. Consultants’ Original Plan 

What was the original plan for the mathematics and language arts technical assistance? 
According to the consultants, the original plan for SIST was to have each consuhant provide 
ten days of technical assistance to the schools they were assigned. During workshops and 
discussion sessions, consultants would lead grade level discussions, model lessons, and 
conduct observations of classroom teachers who tried the materials and processes that 
consultants recommended. 



B. Accomplishments and Changes in Plans 

What was accottmlished bv SIST? In the main, consultants’ responses to this question were 
general descrptions. The consultants provided a general accounting for; days of work at each 
school (day 1, day 2, etc.); narrative descrption of conditions at each school (e.g., materials, 
work arrangements, and school personnel); description of workshops that consultants 
presented alone or in association with other SIST consultants; and description of issues 
encoimtered (e.g., lack of substitutes, saow days to make up, response of teachers and 
primeval, shortage of materials and equipment). 

The consultants produced a general goal and a plan or outline of possible activities to apply 
to all buildings. The inq)lementation of this plan was contingent on recommendations and 
feedback from the teachers. There was also a preliminary plan for the language arts 
component that included seven steps, topics for discussion, and follow-up activity. Analysis 
of data collected from teachers and administrators indicates that these partic^ants became 
aware from the be ginnin g that a main weakness of SIST was the absence of a clear, well- 
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organized plan, coirq)lete with specific objectives that were targeted to the needs of each 
building. 

The content of various documents prepared by the consultants suggests that most of the 
consultants’ work was a prorrq)t and more or less effective response to “needs” identified by 
different teachers during discussion sessions, or it was the consultants’ regular use of standard 
and, in some instances, time-worn workshops. In the previous two sections in this analysis 
of data, there are numerous critical comments that CRE received on surveys from teachers 
and princ^als to support these conclusions about SIST: workshops frequently entailed 
mundane topics and the consuhants’ style appeared to be hit-and-miss and informal, rather 
than an up-to-date, data based, and strategic approach. 

Nevertheless, Linda Ball’s final report includes substantial information on the identification 
and assessment of resources — ^including personnel — at district, school, and grade levels. 
Likewise, the final reports from Pamela Barker Jones and Mj Terry provide a specific and 
thoughtfiil analysis of building-level conditions, including recommendations for the next 
school year. These dociunents he^ to satisfy one of SIST’s inq)ortant commitments to the 
district, namely, to analyze staffing e?q>ertise and resoiuces at the seven identified schools. 
Central office administrators and building princ^als should find this dociunentation to be very 
usefiil\^enplanning activities or forpiuchasing materials for the 1999-2000 school year (For 
details, see list of recommendations at the end of this section). 

If there were significant changes in operations relative to the original plans, why were these 
changes necessary? The most significant issue reported by consultants was that substitute 
teachers were either too few in numher or they were not available when requested. Thus, 
without substitute teachers, consultants could not free the regular classroom teachers from 
their work responsibilities to observe the consultant teaching another classroom or to engage 
in discussions with other teachers about the workshops or their practice. 

Additionally, when a siow day occurred, consultants freed make-up days without substitute 
teachers. Also, Waterbury’s five-day rotating schedule was an issue for the consultants, 
because they were working in the buildings only on certam days, and not every day of the 
week. Hence, if school was canceled because of a mow day, all ofMonday ’ s special activities 
were rescheduled for Tuesday, rendering the consuhant’s plans for Tuesday unworkable. 

This recurrent issue with the substitutes stems, at least in part, from SIST’s organizational 
plan, v^ch was to have the consultants work in the elementary schools for only ten days and 
that these days would be spread from January to June, 1999. If the plan had, instead, been to 
provide an intensive, sustained technical assistance program, taking place every day over two 
or three weeks — as recommended by some principals — ^then the consuhants would have been 
in a better position to respond to exigencies irtq)osed by weather conditions. There also would 
have been a greater opportunity for building trust between consuhant, teachers, and the 
principal, and greater opportunity for building consistency and depth into the SIST initiative. 
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C. Overall Effectiveness of SIST 

How would you assess the effectiveness of the mathematics and language arts components 
of this technical assistance initiative? Consultants reported that it was difficult to assess the 
effectiveness of SIST’s mathematics and language arts conq)onents. The consuhants said that 
they would defer their assessmmt of SIST’s overall effectivraiess because they e?q)erienced 
difficulty with conq)leting some of the planned activities, namefy observation of teachers’ 
follow-up activity and team teaching with the staff to determine the extent to w^ch 
workshops influenced teachers’ regular classroom practice. Consultants reported that the 
teachers tended to resist consuhants’ observation of their teaching activity. Also, consultants 
reported receiving responses about their performance from teachers and princ^als that varied 
from highly appreciative to highly crhicaL However, there was no indication from the 
consuhants about the extent or percentage of work that was conq>leted relative to the plan. 

Review of consuhants’ documents and analysis of comments from teachers and princ^als 
suggests that another source ofthe problem with consultants’ self-assessment may be the lack 
ofstrateglc, building-level plans, based upon substantive prior analysis of student performance 
data, curriculum materials, and teachers’ history of professional development. One consuhant 
identified inadequate preplanning as a substantive issue in regard to SIST’s m:q>lementation. 

D. Issues 

What are the main issues with the Waterbmv SIST? Consuhants identified the following 
issues associated with inq)lementing SIST in the sev«i elementary schools: 

• lack of commitment from the administration at building and district levels 

• poor communication between the district and the Connecticut Academy 

• poor advance planning between the Connecticut Academy, consuhants, princ^als, 
and teachers 

• inconsistency of SIST’s scheduling and operations, due to issues with substitute 
teachers, show days, late openings, and conq)eting programs 

• grade level meetings were not mandatory for all teachers 

E. Recommendations 

What recommendations can you offer for improvement? Consuhants provided the following 
recommendations for inq)roving SIST ’ s operations in the Waterbury PubHc Schools for 1 999- 
2000: 

District 

• Hire a K-8 mathematics coordinator. 

• Increase commhments and responsibiUties ofteachers and administrators in regard to 
professional development. 

• Identify and use lead teachers for curriculum planning and professional development. 

• Inq>rove availability and scheduling of substitute teachers. 

• Place higher priority on purchase of relevant classroom materials. 
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SIST 

• schedule teachers’ inservice activity before the school day begins, after the school day 
ends, and during the sunnner months; avoid inservice activity that pulls elementary 
teachers out of classrooms dimng regular teaching hours. 

• Focus operations on fewer schools and only for regular classroom teachers. 

• Design the program so that the consultant is in the school on a sustained, regular, 
weekly or daily schedule, not on an intermittent schedule. 

• Organize workshop s by grade levels because the needs and interests of primary school 
teachers and intermediate level teachers are very different. 

4. Discussion of Consultants* Activity 

At the beginning of this initiative, representatives from the Connecticut Academy — including 
Gemma Joseph Lunq)kin, consultants, and the evaluator — ^held an introductory meeting with 
the assistant superintendent and all seven of the building principals for approximately two 
hours. During this meeting, the assistant superintendent introduced everyone present, 
explained the interests of Waterhury Public Schools regarding improvement of students’ 
academic performance, and welcomed assistance from the Connecticut Academy. 

Ms. Luirq)kin provided an overview of SIST’s operations and solicited comments from 
everyone present. Building principals raised questions about the schedule, availability of 
substitute teachers, and details of the SIST program.. As a follow-up activity, Ms. Lumpkin 
and the consuhants met with each of the building princ^als for two hour meetings to provide 
a more detailed e?q)lanation of SIST’s operations and the implications for their buildings. 
Additionally, the SIST consultants attended faculty meetings in an effort to explain the plan 
to all elementary teachers and to solicit their feedback. 

Ms. Lunqikin reported that SIST consultants spent approximately twelve days at each of the 
elementary schools and worked to cover every grade level. However, she noted that in the 
future, she would recommend that more days be set aside for SIST. Additionally, she 
recommended that SIST focus on one or perhaps two grade levels at each school, depending 
on the size of the school, instead ofall elementary grades. Also, Ms. Limq)kin recommended 
that SIST devote more time to intensive p lanning at the individual schools, in order to design 
the program to meet the needs of each building and to he^ resolve scheduling problems. 

Ms. Lmrqikin noted that the availability of substitute teachers was a great hindrance to SIST’s 
original inq}lementation plan. The lack of substitutes, she concluded, severely hindered our 
ability to reach the goals and objectives of the original plan. As a result, the plan was reduced 
to classroom presentations and limited group sessions. 

Two consultants, Mj Terry and Pamela Barker Jones, provided docxunentation in their reports 
regarding their review of the CMT data, in preparation for or during delivery of their SIST 
workshops. Reports from the other SIST consultants either do not mention the review of 
CMT data or, in one case, make several references to the inq)ortance of teachers using the 
CMT goals and objectives A^iien planning to teach. 
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However, the documentation provided by the consultants regarding their decision- making 
process suggests that the procedures for data based decision-making for this technical 
assistance program have not been established beforehand by the consultants or by the 
Connecticut Academy. Otherwise, if these procedures were in place beforehand, they were 
not apphed consistently nor rigorously by the consultants. 

Analysis of data from the surveys and interviews reveals that teachers and administrators 
identified planning, personnel, and organization as the three main weaknesses of the SIST 
initiative. 

Thus, CKE recommends that the Connecticut AcadeitQ^ establish definitions, parameters, 
materials, and routine procedures for consultants to use A\fren providing on-site technical 
assistance on CMT data based decision-making to teachers and administrators. The result 
should not be so structured as to be an obstacle to in^roving a building’s instructional 
program. However, there is nevertheless a real need for an intervention program to have a 
defined, effective, and working system for data based decision-making and follow-up 
activity — ^including property ahgned and consistent curriculum development, assessment, and 
professional development. 

Additionally, CKE recommends that the Connecticut Academy develop a working model for 
budding-level processes, operations, and decision-making, with specific reference to roles of 
all significant partictyants, including regular classroom teachers, lead teachers, specialists, 
princtyals, central office administration, substitute teachers, and consultants. 
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LESSONS LEARNED 
The tnatn lessons learned in this initiative are: 

• the individual school building, stafi^ students, and piinc^al must constitute the focal 
unit for SIST’s planning, design, scheduling, operations, personnel, and evaluation; 

• planning must begin well in advance of the first day of school — during the preceding 
summer or ^ring of previous school year; 

• piinc^als and lead teachers must be directly involved in SIST’s planning, 
development, and — ^when feasible— its operations, in order to promote collaboration 
at all levels in the school’s organization, mutual ownersh^, and accountability; 

• public announcements about the initiative should occm after staff and administration 
have been alerted to the decision to include SIST in a building’s work schedule; 

• consultants must have strengths in areas of critical need, including expertise with 
teaching in specific grade levels and with academic skills , and they must provide 
workshops and other services that are valued highly by the teachers and piinc^als; 

• the School Board, central office, teachers, piinc^als, consultants, and the Connecticut 
Academy must see SIST as a long-term, intensive, thoughtftil, and process-oriented 
approach to school in^rovement; 

• patience, trust, consistency and coherence, determination to succeed, and 
accountability are critically inq)ortant values to assure ^ort-term and long-term 
success — ^trust between teachers, piinc^als, and consultants is essential 
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SUMMARY 

The followmg chart presents a summary of CRE’s responsibilities to the Waterbury Pubfic 
Schools and to the Connecticut Academy. Also included in this chart are the status of each 
task at the end of the term and a brief statement regarding the results of each task. 



TASK 


STATUS 


RESULT 


Assess the overall performance of the 
existing instructional program 


Completed 


Documentation shows schools need 
ongoing strategic technical assistance. 


Measure the effectiveness of the SIST 
concept 


Completed 


Concept is sound However, SIST will 
benefit from improving: planning, 
organisation, scheduling, communications, 
and persormel. 


Profvide specific, written performance 
measures and benchmarics for subsequent 


Completed 


See analysis of data and recommendations 
in earlier sections of this report and below. 





This evaluation study of the SIST initiative in the seven identified elementary schools of the 
Waterbiuy Public Schools demonstrates that a data based approach to decision-making is a 
highly useful, strategic approach to educational programming. It is a procedure that 
Waterbury’s school administrators and teachers prefer, rather than continuing to rely on 
obsolete methods. Also, the first-year in^lementation of a data based decision-making 
process — like any other inq)ortant and useful innovation — entails con^>lications, necessitates 
refinements, and requires time and consistent application, in order to realize success. Thus, 
it is not surprising that the analysis of data shows mixed resuhs overall. 

From Waterbiuy’s teachers and administrators, SIST received an overall 60% performance 
rating. Similarly, consultants from the Connecticut Acaden^ expressed disappointment that 
the program had not gone as smoothly as they had expected. However, the SIST idea for 
providing technical assistance to elementary schools is recognized by all the educators as a 
very sound approach to inq)roving students’ learning, in general, and for inq)roving their 
performance on the CMT, in particular. Despite the issues regarding SIST’s in:q)lementation, 
it is nonetheless a program that will benefit the district. Thus, the assistant superintendent 
^ould be commended for his foresight and risk-taking. 

The analysis of data reveals that teachers and administrators want qiiality technical assistance 
with professional development. Additionally, they recognize that the usual way of providing 
inservice — casual programs, top-down decision-making, or picking and choosing from the 
latest feds — ^is not working to the benefit of students or teachers. Thus, everyone agrees that 
a high quality, strategic approach, based upon analysis of student CMT performance data, 
offers the best and the most up-to-date approach for resolving Waterbury’s continuing issue 
of low student performance data with basic skills. 
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Data analysis ^ows that SIST got off to a rocky start with the assessment conq)onent, 
because the consultants used a presentation style that was offensive to the local educators. 
Additionally, documentation fromteachers and administrators shows that the implementation 
of math conq)onent was highly satis&ctory to the educators in two schools, modestly 
satis&ctory to many teachers and administrators in the other five schools, and not satisfactory 
to a small er group of teachers and administrators across the seven schools. The language arts 
con^onent received low marks from a majority of partic^ants. 

Educators who found SIST to be satis&ctory recognized two math consultants as the source 
of their conclusion about SIST’s overall effectiveness, hi the opioion of these teachers and 
adininistrators, the two consultants presented interesting and useful workdiops. Nevertheless, 
the satisfied educators provided a list of critical comments about the SIST initiative that 
matched very closely the criticism that CRE received from other educators, who were not 
satisfied with the program Additionally, the satisfied educators said that, if SIST revises its 
program consistent with their recommendations, they might look positively on its continuation 
in their school in the futme. 

Educators who were not satisfied with SIST reported that the assessment conoponent, the 
math and language arts con^onents, and/or the language arts conq)onent were the source of 
their disappointment. The main issues for these individuals were that the consultants used out- 
of-date material, they were not &miliar with grade level differences, they did not coordinate 
their inservice with building needs, and they gave lackluster presentations. The ongoing 
problem with too few or no substitute teachers was fiustrating for everyone concerned with 
this initiative, including the consultants. Many unsatisfied educators said that they would 
prefer to have grade level meetings than to devote time and money to ineffective consultants. 

CRE’s evaluation of SIST shows strong agreement between teachers, administrators, and 
SIST consuhants regarding the strengths of this initiative, its start-up problems, and the best 
solutions to those problems. Educational innovations always encounter resistance from 
various sources, suffer setbacks, and have to regroup or to reorganize for a more effective 
approach to the situation — especially in their first year of operations. Thus, CRE strongly 
recommends that both parties to this contract e?q)loit what works best and continue working 
together to inq>rove teaching and learning in Waterbruy’s elementary schools. At the 
conclusion of this report, there are specific recommendations for improvement. 

At this time, the Connecticut Academy and the Waterbury Public Schools should collaborate 
to clarify SIST’s purpose and to establish rigorous and specific objectives, processes, and 
standards for the initiative. In s umme r 1999, a year-long technical assistance program should 
resume in two or three of the seven original elementary schools. By fell 1999, the revised 
SIST program should attain higher approval ratings from teachers and principals. It ^ould 
also provide greater assurance of success for teachers and students. Once the Waterbury 
model of technical assistance has achieved definition and has demonstrated its effectiveness 
in these schools, it should be applied with appropriate adjustments on a scheduled basis in 
other elementary schools in Waterbiuy. 
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la conchisioiL, the lessons learned are the best resuhs of this initiative. Patience, mutual trust, 
consisten(^, coherence, accountability, and determination to succeed must continue as the 
basic princ^les guiding SiST’s development. The Connecticut Academy’s reputation for 
providing quality service and materials to inq)rove mathematics and science education is well- 
recognized statewide. Administrators and teachers in the Wateibury Public Schools egress 
a commitment to inq>roving students’ CMT performance through the strategic professional 
development of elemoitary school teachers. Furthermore, everyone involved with this 
initiative recognizes that strategic p lanning and operations — ^informed by a current and e7q)licit 
analysis of essential data — are critical elements for inq>roving the school’s curriculum, 
assessmoit practices, and professional developmoit. With, these recommendations in mind, 
CRE strongly urges continuation of SIST in the Waterbury Public Schools. 
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RECOMMENDATIONS 

The following recommendations come from surveys and interviews with W aterbury ’ s teachers 
and principals and from reports and meetings with the consultants, vsfro worked for the 
Connecticut Academy. Additionally, CRE provides a number ofrecommendations for analysis 
of CMT data and SIST program changes, in order to support an effective technical assistance 
program. 

1. Analysis of CMT Data 

• Analyze CMT data by cohort groups of students within buildings and, to the extent 
possible, within the district, in order to study the long-term impact of intervention 
strategies and to assess the merit of different intervention strategies. 

/ 

• Hire a consultant — ^v\fro is familiar with CMT data, urban education issues, 
curriculum, instruction, and assessment — ^to develop graphical representations and 
related materials pertaining to building-level student performance data for use by 
teachers and principals A^en p lanning for technical assistance. 

• Focus on the K-5 students who do not change schools, using as the minimum criterion 
of success for the SIST program 100% of these students scoring at or above state 
goal Develop a similar plan for all other childrai, based on recommendations from 
the study of student mobility issues in Waterbury. 

• Incorporate appropriate performance-based assessment instruments to monitor all 
children’s development of specific skills in grades K-5, including relevant follow-up 
methods and materials for developmental and remedial instruction. 

• Use the yearly analysis of CMT data (as suggested in this report) to identify specific 
skills for special age/grade appropriate enq>hasis in each grade K-5, using 85 percent 
mastery (state goal) on all performance measures as the main criterion of success. 

2. SIST Program 

• Maiutain regular commmucations with central office administrators in regard to 
SIST’s operations, in general, and to feedback from participants, specifically. 

• Enq>hasize \^frat works. Demonstrate to teachers the methods and materials that help 
urban children to learn reading, writing, and/or mathematics. 

• Organize and facilitate a sufficient number of discussion sessions, during which 
teachers describe common issues and propose effective solutions. 

• Monitor on a daily basis the activities and processes of consultants, in order to ensure 
quality performance, including pronq)t response to issues and problems. Set the goal 
for each building at 100%partic^ant satisfrctionthat SIST provides excellent service. 
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• Consult with teachers and principals at the earliest opportunity to determine building 
expectations and needs, and continue consulting on a regular basis throughout the 
initiative. 

• Focus on intensive, strategic, age/grade appropriate developmental programs in math 
and reading/study skills for Pre-K throu^ third grade. Set 85 percent mastery of 
specific skills (state goal) as the criterion of success. 

• Provide follow-up developmental and remedial programs in math and reading/study 
skills for fourth and fifth grade students. Set 85 percent mastery of ^ecific skills 
(state goal) as the criterion of success. Monitor closely for age-appropriate 
development and adjust instruction based on students’ needs. 

• Transition district administrators, piinc^als, and lead teachers within three years to 
fiill re^onsibility for analysis of CMT data and related documentation, in order to 
support data based decision-making at district and biiilding levels. 

• Hire consultants to study and report on student mobility issues in Waterbury as they 
relate to curriculum, instruction, and assessment. Develop recommendations for 
policy changes, general program development, and intervention strategies in order to 
mitigate the adverse effects of extremely high student mobility rates. 

3. Personnel 

• During 1999 summer, hire a consultant to study CMT data and to work with district 
administrators, piinc^als, and lead teachers to identify ^ecific skills as objectives for 
SIST. Select methods, materials, equipment, and personnel to accomplish the 
objectives. 

• Kre exemplary technical assistance providers, with qualifications — ^including practical 
experience — in pre-kindergarten and kindergarten and in grades 1, 2, 3, 4, and 5. 

• HSre a consultant for the assessment component, >^lio has substantial, direct 
experience with analysis of CMT data for data based decision-making, and wlio has 
the skill to work effectively with teachers and administrators. 

• Lisist on high quality performance by all consultants and ensure fit of personnel and 
program with individual schools. 

• Inprove availability and scheduling of substitute teachers. 
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4, Operations 

• Conduct intensive study of CMT data and other relevant documentation, in order to 
prepare a clear, formal, consistent, and data based plan for professional development 
that is specific to each building. 

• Diss emina te the technical assistance plan to the teachers and princ^al, in order to 
support direct, open, and continuous communication with all partic^ants. 

• Provide sustained, consistent, strategic technical assistance (have princ^al and lead 
teachers help determine the time finme and schedule of operations). 

• Organize consuhants’ work to include presenting workshops that are consistent with 
the plan, collaborating with teachers, working hand-in-hand with regular classroom 
teachers, serving as grade level resource teachers, and identifying lead teachers to 
assist with in^lementing the program 

• Organize workshops by age/grade levels, because the needs and interests of pre- 
kindergarten, kindergarten, primary school, and intermediate level teachers are very 
difiTerent. 

5. Evaluation and Assessment 

• Continue the independent external evaluation of the SIST initiative, in order to 
describe, measure, and assess its performance during this formative period. 
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BIOGRAPHICAL SKETCHES 

Curriculuin Research and Evaluation is a firm that is devoted to research and development 
of programs in the field of education. CRE’s specialities are: (1) to provide services in order 
to evaluate the quality of education programs for private business and industrial conq)anies, 
public and private fimding agencies, and schools; and (2) to develop and guide the 
inq)lementation of curriculum and instruction. 

Charies Bruckerhoff is Principal Evaluator and Research Associate for Curriculum Research 
and Evaluation. He received his doctorate from the University of Wisconsin. Ms research 
interests are curriculum theory and development, philosophy of education, effects of public 
policy on the classroom teacher, and school restructuring. He is the author of Between 
Classes: Faculty Life at Truman High and has written articles on ciuiiculum development, 
qualitative research, urban collahoratives, and disadvantaged youth. 

Theresa Bruckerhoff is Operations Manager and Research Associate for Curriculum 
Research and Evaluation. She has a B.S. in Elementary Education and a M.S. in Curriculum 
and Instruction. She has sixteen years of teaching e?q)erience ranging from preschool to the 
middle school levels. She taught in gifted programs, fecial education programs, and is an 
experienced classroom teacher. Most recently ^e hasheld executive hoard positions for child 
care centers and a nursery school Currently, she studies state and national programs for 
teachers’ professional development and school restructuring. 
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Interview with Building Principal 
Interview with Assistant Superintendent 
Waterbury Teachers Formative Evaluation 
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Waterbiiry Public Schools 
and 

Connecticut Academy for Education 
SIST 

BVTERVIEW WITH ASSISTANT SUPERINTENDENT 

1. Are the mathematics and language arts auricula in Waterbury elementary schools con^rehen^e, that is, 
developed fiilfy for all grades? 

Mathematics O Yes O No 

If no, please e>q)lain. 



Date: 


Sdiool: 


Asst SqperintendfntI 


Timec 







Language Arts O Yes O No 
If no, please e?q)lain. 



2. Are the elementary mathematics and language arts curricula based on the Curriculum Frameworks developed 
by the Connecticut State Department of Education? 

Mathematics O Yes O No 

If no, please e?q)lain. 



Language Arts O Yes O No 
If no, please e?q)lain. 



3. Are the elementary teachers’ instructional practices in mathematics and language arts consistent with the 
curricula in each of these areas? 

Mathematics OYes ONo 

If no, please e?q)lain. 



Language Arts O Yes O No 
If no, please e7q)laiiL 



4. What percentage of elementary &culty transferred out of schools during; 

1996- 97? % 

1997- 98? % 

1998- 99? % 

5. How would you assess current student performance m mathematics, as measured by the CMT? 
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6. How would you assess current student performance in language arts, as measured by the CMT? 

7. What is your overall assessment of SIST? In other words, how effective was this professional development 
program at meeting elementary teachers’ instmctional needs? 

8. Did SIST acconq)iyi its purposes or objectives at a satis&ctory level? O Yes O No 
If no, please e?q>lain. 



9. To w^at extent did the mathematics conq)onent acconq)lish its purposes? 



10. To w^t extent did the language arts con^onent accon^lish its purposes? 



1 1. How would you describe the teachers’ and princ^als’ response to the mathematics conq)onent of SIST? 



12. How would you describe the teachers’ and princ^als’ response to the language arts conq)onent of SIST? 



13. What are the most valuable (Le., useful) a^ects of SIST? 



14. What, if any, are the weaknesses or issues associated with SIST? 



15. Win you refy on SIST for Waterbiuy’s elementary level professional development next school year? 



16. What reconnnendations can you offer to inq)rove the SIST program for the Waterbiuy Public Schools? 



O 



Cumculuzn Research & EvahiadoL. 237 SingleUn Road, CT 06235#S6CM55-2339#Fax 860-455-001 14www.creus.oam 



Waterbary Public Schools 
and 

Connecticut Academy for Education 
SIST 

BVTERVIEW WITH BUILDING PRINCIPAL 

1 . Are the mathematics and language arts curricula in your building con^rdiensive, that is, developed fully for 
an grades? 

Mathematics O Yes O No 

If no, please e?q>lain. 

Language Arts O Yes O No 
If no, please e?q>lain. 

2. Are the mathematics and language arts curricula based on the Curriculum Frameworks developed by the 
Connecticut State Department of Education? 

Mathematics OYes ONo 

If no, please e7q>lain. 

Language Arts O Yes O No 
If no, please explain. 

3. Are the teachers’ instmctional practices in mathematics and language arts consistent with the curricula in 
each of these areas? 

Mathematics O Yes O No 

If no, please e7q>lain. 



Date: 


SdiooL 


Pcindpal! 


Time: 







Language Arts O Yes O No 
If no, please explain. 



4. What percentage of your focuhy transferred out of schools during; 

1996- 97? % 

1997- 98? % 

1998- 99? % 

5. How would you assess current student performance in mathematics, as measured by the CMT? 
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6. How would you assess current student performance in language arts, as measured by the CMT? 

7. What is your overall assessment of SIST? hi other words, how effective was this professional development 
program at meeting elementary teachers’ instructional needs? 

8. Did SIST accon^lish its purposes or objectives at a satis&ctoiy level? O Yes O No 
If no, please eiqilain. 



9. To what extent did the mathematics con^onent accon^Hsh its purposes? 



10. To what extent did the language arts conqionent accon^lish its purposes? 



1 1. How would you describe the teachers’ response to the mathematics component of SIST? 



12. How would you describe the teachers’ response to the language arts con^onent of SIST? 



13. What are the most valuable (Le., useful) aspects of SIST? 



14. What, if any, are the weaknesses or issues associated with SIST? 



15. Will you rely on SIST for your teachers’ professional development next school year? 



16. What recommendations can you offer to inqjrove the SIST program for the Waterbury Public Schools? 



O 
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Education Program Improvement 

Waterbary Public Schools 
and 

Connecticut Academy for Education 

TEACHER’S FORMATIVE EVALUATION 



Dear Professional Educator ^ 

Curriculum Research ^Evaluation (CRE) is an independent external evaluator with responsibilityfor assessing the Education Program 
Improvement initiative by the Connecticut Academy for Education and the Waterbwry Public Schools. Please take a few minutes to 
complete this evaluation of the School Improvement Support Team (SIST) in your building. Use the SASEfor its retioji to CRE. Your 
professional opinion is very important for guiding the project *s current plans and for measuring its overall ejfectiveness. All participants 
in this study will be anonymous and all contributions will remain confidential. Thank you very much in advance for your cooperation 
and assistance. 

Sincerely, 

Charles Bruckerhqff 
Princi pal Evaluator CRE 

Demosrayhic Information 

Building assignment: 

Position: □ Teacher □ Administrator 

Gender: □ Female □ Male 

Grade(s) currently assigned: □ Pre-K DK □! D2 D3 D4 D5 D 6 □? D8 

Area(s) of concentration (if appropriate): 

Highest degree: □ BA/BS □ MA/MS □ MA/MS +15 credits □ Doctorate □ Pursuing advanced graduate degree 
BEST certified: □ Yes □ No 

Other certifications: 

Years of teaching Years of administration (if appropriate): 

Level of Particioation 

Indicate how many and vd\Bt kind of initial, small SIST meetings you attended Number of Meetings Grade Level 



How many SIST workshqjs for math did you attend? O Not ^jplicable 

How many SIST woricshops for language arts did you attend? O Not ^>pUcable 

How many times did you take the opportunity to watch the SIST math consultant teach? O Not ^>plicable 

How many times did you take the ORX)rtunity to watch the SIST language arts consultant teach? O Not a^licable 

How often did you meet with the consultants to discuss the workshop's implications for your classroom teaching? 



Overall Assessment of SIST Workshops 

The SIST workshoDS were; 


VcvyStrtMigjiy A^«e Disagree 
Agree 


VcryStrongh' 

DUiqpee 


NoOptiriflD 


Up-to-date? 


o 


o 


o 


o 


o 


Timely? 


o 


o 


o 


o 


o 


Well-organized? 


o 


o 


o 


o 


o 


Well-managed? 


o 


o 


o 


o 


o 


EEfectively implemented? 


o 


o 


o 


o 


o 


Addressed wy instructional needs? 


o 


o 


o 


o 


o 


Addressed the instructional needs of other teachers in this building? 


o 


o 


o 


o 


o 


How would you assess the overall effectiveness of these worksht^? O Ezcdient O Good O Average 

Describe your overall impression cf the SIST math and language arts workshops as separate entities. 


O Poor O No Opinioo 



BEST COPY AVAlUaiBLE 

(Over please) 



o 
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SIST Consultant’s Effectiveness 


Vciy Strong 


Agree 


Ungiree 


V«fyStron0y 


NoOiUai 




Agree 






DInqpree 




The SIST consultants were: 












Effective with presentation skills? 


o 


o 


o 


o 


o 


Enthusiastic about the topics? 


o 


o 


o 


o 


o 


Eager to help y<Hi? 


o 


o 


o 


o 


o 


Well-prepared, in regard to relevant content knowledge? 


o 


o 


o 


o 


o 


Equi{q)ed with good teaching ideas — for your purposes? 


o 


o 


o 


o 


o 


Effective with classroom modeling? 


o 


o 


o 


o 


o 


Effective v/hen interacting with the variety of people in the school? 


o 


o 


o 


o 


o 


Effective v/hen demonstrating use of manipuladves for teaching? 


o 


o 


o 


o 


o 


Effective when demonstrating use of instructional technology, such as calculators? 


o 


o 


o 


o 


o 


Effective vsdien adapting to changing schedules and other issues at the building? 


o 


o 


o 


o 


o 


Impact on Teaching 


Vciy Strong 


Affce 


DUa^ee 


Very Strongjly 


N60|g]ioo 




A^ee 






Dbagree 




The SIST workshoDs: 












Were filled with new methods and materials? 


o 


o 


o 


o 


o 


Contained methods and materials that you alreacfy knew. 












but they were presented in a fresh and more imderstandable format? 


o 


o 


o 


o 


o 


Have you started using some of the methods and materials you encountered in the workshops? 


O Yes 


ONo 




How much of the SIST methods and materials do you plan to use next year? O 100% O 75% 


O 50% O 25% O None 


Assess the workshops’ contribution to your professional development O EiccOent 


OCood 


O Average OPoor O No Opinion 



Impact on Students^ Learning 

What do you expect will be the most important contribution(s) to students" learning fiom your use of SIST wcnicshop materials and 
methods? 



Do you have any evidence, thus far, that these methods and materials are improving students’ achievement? O Yes O No 
Please explain the impact on students’ learning. 



How do students respond to your use of the SIST workshop methods and materials? 



Contributions to Buildini^Level Improvement 
The SIST workshops: 

Increased building-level expectations regarding professional development? 
Created a very favorable response among aU teachers? 

Created a very favorable response from the administration? 

Created a strong expectation to continue SIST workshops in the future? 
Assess the overall building-level response to SIST workshops. O Eicdient 

Should SIST workshops continue in the immediate future? O Yes O No 
Please explain your response to continuing SIST worksh(q>s. 



Vciy Strang Agree Dln^ve VcnrStm^ NoOptarioi 
Agree Disagree 

o o o o o 

o o o o o 

o o o o o 

o o o o o 

O Good O Average O Poor O No OididoB 



BEST COPY AVAILABLE 



(Next please) 
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Overall Assessment of Mathematics Workshops 

1. What are the strengths of the School Improvement Support Team (SIST)? 



2. What, if any, are the weaknesses, shortcomings, or outstanding issues of SIST? 



3. What specific recommendations can you offer for improving SIST in these areas: 
Plan for Professional Development 

Operations and Performance 

Scheduling 

Other 



Overall Assessment of Language Arts Workshops 

1. What are the strengths of the School Improvement Support Team (SIST)? 



2. What, if any, are the weaknesses, shortcomings, or outstanding issues of SIST? 



3. What specific recommendations can you offer for improving SIST in these areas: 
Plan for Professional Development 



Operations and Performance 



Scheduling 



Other 



O 
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